Network Analysis : Link Prediction with NetworkX
Prepared by Alex Aw
16 March 2023

Table of Content

  1. Objectives and Background
  2. EDA & Preprocessing
    2.1 Visualisation
    2.2 Further Pre-processing
  3. Centrality Features
    3.1 Building the features
    3.2 Building the classification model
  4. In-Built Algorithm
    4.1 Jaccard Coefficient
    4.2 Adamic-Adar
    4.3 Preferential Attachment
    4.4 Plotting the results
  5. Afterthought
    5.1 Which Algo?
    5.2 What's next?

Objectives and Background ¶

To practise Network Analysis using The Marvel Universe Social Network data. Following some of fellow contributors data preprocessing to gain an understanding how to work with network graph data. The notebooks in Marvel showed good network analysis but did not apply Link Prediction.

Using references from other sites and examples, this notebook is an attempt to apply Link Prediction. 2 methods have been found using NetworkX to generate the features or in-built algorithm to do the link predictions. The first uses NetworkX to generate different features based on centrality and then proceed to use classifying algorithm to predict the outcome [1]. The second uses NetworkX built in algorithm such as Jaccard Coefficient, Adamic-Adar and Prederential Attachment to predict the outcome [2]. So I'll try both methods and see the result and analyse the performances.

A third and possible alternative would be to consider using Graph Neural Network (GNN). To attempt GNN, I'd use the same dataset but on another notebook.

[1] NetworkX Link Prediction
[2] Graph Learning

In [1]:
'''
There is version conflict between networkx and scipy. Error arose when trying to draw the graph or calculate the centrality.
"AttributeError: module 'scipy.sparse' has no attribute 'coo_array'"

After checking online and testing several version mix, scipy latest version (as of writing) 1.10.1 and 
networkx version 2.6.3 worked for me.
'''
# !pip install scipy==1.10.1
# !pip install networkx==2.6.3
Out[1]:
'\nThere is version conflict between networkx and scipy. Error arose when trying to draw the graph or calculate the centrality.\n"AttributeError: module \'scipy.sparse\' has no attribute \'coo_array\'"\n\nAfter checking online and testing several version mix, scipy latest version (as of writing) 1.10.1 and \nnetworkx version 2.6.3 worked for me.\n'
In [2]:
import pandas as pd
import numpy as np 
import matplotlib.pyplot as plt
import networkx as nx
%matplotlib inline 
from IPython.display import display, Image
from PIL import Image
import seaborn as sns
import itertools
import pickle
import random
from tqdm import tqdm

from sklearn.model_selection import train_test_split 
from sklearn.ensemble import GradientBoostingClassifier  
from sklearn.metrics import classification_report, confusion_matrix, balanced_accuracy_score
from sklearn.metrics import roc_curve, plot_roc_curve, roc_auc_score, accuracy_score

from lightgbm import LGBMClassifier
In [3]:
df_edge = pd.read_csv('data/edges.csv') # Edges
df_hero = pd.read_csv('data/hero-network.csv') # Hero
df_node = pd.read_csv('data/nodes.csv') # Nodes

Top

EDA & Preprocessing ¶

In [4]:
print(df_edge.head(10))
print(df_hero.head(10))
print(df_node.head(10))

                   hero     comic
0  24-HOUR MAN/EMMANUEL    AA2 35
1  3-D MAN/CHARLES CHAN     AVF 4
2  3-D MAN/CHARLES CHAN     AVF 5
3  3-D MAN/CHARLES CHAN     COC 1
4  3-D MAN/CHARLES CHAN    H2 251
5  3-D MAN/CHARLES CHAN    H2 252
6  3-D MAN/CHARLES CHAN  M/PRM 35
7  3-D MAN/CHARLES CHAN  M/PRM 36
8  3-D MAN/CHARLES CHAN  M/PRM 37
9  3-D MAN/CHARLES CHAN     WI? 9
                  hero1                 hero2
0         LITTLE, ABNER        PRINCESS ZANDA
1         LITTLE, ABNER  BLACK PANTHER/T'CHAL
2  BLACK PANTHER/T'CHAL        PRINCESS ZANDA
3         LITTLE, ABNER        PRINCESS ZANDA
4         LITTLE, ABNER  BLACK PANTHER/T'CHAL
5  BLACK PANTHER/T'CHAL        PRINCESS ZANDA
6  STEELE, SIMON/WOLFGA      FORTUNE, DOMINIC
7  STEELE, SIMON/WOLFGA   ERWIN, CLYTEMNESTRA
8  STEELE, SIMON/WOLFGA  IRON MAN/TONY STARK 
9  STEELE, SIMON/WOLFGA  IRON MAN IV/JAMES R.
                   node   type
0               2001 10  comic
1                2001 8  comic
2                2001 9  comic
3  24-HOUR MAN/EMMANUEL   hero
4  3-D MAN/CHARLES CHAN   hero
5      4-D MAN/MERCURIO   hero
6               8-BALL/   hero
7                 A '00  comic
8                 A '01  comic
9                 A 100  comic

Cleaning up the text by limiting text character to 20 and split on "/" since the text after / is name of the hero.

In [5]:
for c in ['hero1', 'hero2']:
    print(f"{c} name max in hero-network.csv : {max(df_hero[c].apply(lambda x : len(x)))}")
    

# Name Preprocessing
## only use len 20 & only use left string bas on "/"
for c in ['hero1', 'hero2']:
    df_hero[c] = df_hero[c].apply(lambda x : x[:20].split("/")[0])
df_edge['hero'] = df_edge['hero'].apply(lambda x : x[:20].split("/")[0])
df_node['node'] = df_node['node'].apply(lambda x : x[:20].split("/")[0])

print(100*"*")
print(df_edge.head(10))
print(df_hero.head(10))
print(df_node.head(10))

hero1 name max in hero-network.csv : 20
hero2 name max in hero-network.csv : 20
****************************************************************************************************
          hero     comic
0  24-HOUR MAN    AA2 35
1      3-D MAN     AVF 4
2      3-D MAN     AVF 5
3      3-D MAN     COC 1
4      3-D MAN    H2 251
5      3-D MAN    H2 252
6      3-D MAN  M/PRM 35
7      3-D MAN  M/PRM 36
8      3-D MAN  M/PRM 37
9      3-D MAN     WI? 9
           hero1                hero2
0  LITTLE, ABNER       PRINCESS ZANDA
1  LITTLE, ABNER        BLACK PANTHER
2  BLACK PANTHER       PRINCESS ZANDA
3  LITTLE, ABNER       PRINCESS ZANDA
4  LITTLE, ABNER        BLACK PANTHER
5  BLACK PANTHER       PRINCESS ZANDA
6  STEELE, SIMON     FORTUNE, DOMINIC
7  STEELE, SIMON  ERWIN, CLYTEMNESTRA
8  STEELE, SIMON             IRON MAN
9  STEELE, SIMON          IRON MAN IV
          node   type
0      2001 10  comic
1       2001 8  comic
2       2001 9  comic
3  24-HOUR MAN   hero
4      3-D MAN   hero
5      4-D MAN   hero
6       8-BALL   hero
7        A '00  comic
8        A '01  comic
9        A 100  comic

In hero-network data, hero1 & hero2 have different values.

In [6]:
print(df_hero[df_hero['hero1']=="CAPTAIN AMERICA"].count())

print(df_hero[df_hero['hero2']=="CAPTAIN AMERICA"].count())

hero1    8149
hero2    8149
dtype: int64
hero1    8350
hero2    8350
dtype: int64
In [7]:
top20_hero = df_hero["hero1"].value_counts().head(20)

top20_hero
Out[7]:
CAPTAIN AMERICA    8149
SPIDER-MAN         6652
IRON MAN           5850
THOR               5712
THING              5369
WOLVERINE          5230
SCARLET WITCH      5184
VISION             5067
HUMAN TORCH        4970
MR. FANTASTIC      4788
INVISIBLE WOMAN    4723
BEAST              4628
HAWK               4506
CYCLOPS            4492
WASP               4452
STORM              4170
COLOSSUS II        3997
PROFESSOR X        3973
ANT-MAN            3727
MARVEL GIRL        3667
Name: hero1, dtype: int64

Top

Visualisation ¶

Creating a 1% sample of the 8 more well known and popular heroes.

In [8]:
thor = df_hero[df_hero['hero1']=="THOR"].sample(frac=0.01, random_state=42)
ironman = df_hero[df_hero['hero1']=="IRON MAN"].sample(frac=0.01, random_state=42)
spider = df_hero[df_hero['hero1']=="SPIDER-MAN"].sample(frac=0.01, random_state=42)
capt = df_hero[df_hero['hero1']=="CAPTAIN AMERICA"].sample(frac=0.01, random_state=42)
hulk = df_hero[df_hero['hero1']=="HULK"].sample(frac=0.01, random_state=42)
storm = df_hero[df_hero['hero1']=="STORM"].sample(frac=0.01, random_state=42)
wolverine = df_hero[df_hero['hero1']=="WOLVERINE"].sample(frac=0.01, random_state=42)
strange = df_hero[df_hero['hero1']=="DR. STRANGE"].sample(frac=0.01, random_state=42)
In [9]:
samplehero = pd.concat([thor,ironman,spider,capt,hulk,storm,wolverine,strange],axis = 0)
samplehero.info()

<class 'pandas.core.frame.DataFrame'>
Int64Index: 422 entries, 152238 to 69526
Data columns (total 2 columns):
 #   Column  Non-Null Count  Dtype 
---  ------  --------------  ----- 
 0   hero1   422 non-null    object
 1   hero2   422 non-null    object
dtypes: object(2)
memory usage: 9.9+ KB

Distribution of heroes in column hero1

In [10]:
plt.figure(figsize = (15,8))
sns.histplot(data=samplehero["hero1"])
Out[10]:
<AxesSubplot:xlabel='hero1', ylabel='Count'>
In [11]:
samplehero["hero2"].value_counts()
Out[11]:
ANT-MAN             8
SCARLET WITCH       8
HUMAN TORCH         7
IRON MAN            6
CAPTAIN AMERICA     5
                   ..
HOPKINS, STEVE      1
CHAMELEON           1
TALISMAN II         1
ROBERTSON, RANDY    1
CLEA                1
Name: hero2, Length: 261, dtype: int64

Distribution of heroes in column hero2

In [12]:
plt.figure(figsize = (15,8))
sns.histplot(data=samplehero["hero2"])
Out[12]:
<AxesSubplot:xlabel='hero2', ylabel='Count'>

A simple network graph based on the 1% sample of the popular heroes.

In [13]:
G = nx.from_pandas_edgelist(samplehero, 'hero1', 'hero2')
pos = nx.spring_layout(G, seed=9666)
options = {
    "node_color": "skyblue",
    "edge_color": "lightgrey",
    "width": 2,
    "edge_cmap": plt.cm.Blues,
    "with_labels": True
#     "alpha": 0.5
}
plt.figure(figsize = (18,18))
nx.draw(G, **options)
plt.show()

Top

Further Pre-processing ¶

Full sample from the 8 popular heroes.

In [14]:
thor = df_hero[df_hero['hero1']=="THOR"]
ironman = df_hero[df_hero['hero1']=="IRON MAN"]
spider = df_hero[df_hero['hero1']=="SPIDER-MAN"]
capt = df_hero[df_hero['hero1']=="CAPTAIN AMERICA"]
hulk = df_hero[df_hero['hero1']=="HULK"]
storm = df_hero[df_hero['hero1']=="STORM"]
wolverine = df_hero[df_hero['hero1']=="WOLVERINE"]
strange = df_hero[df_hero['hero1']=="DR. STRANGE"]

samp_pophero = pd.concat([thor,ironman,spider,capt,hulk,storm,wolverine,strange],axis = 0)
samp_pophero.reset_index(inplace=True, drop=True)

samp_pophero.head(10)
Out[14]:
hero1 hero2
0 THOR GRANT, GLORIA GLORY
1 THOR FAIRMONT, HANNAH
2 THOR PARKER, MAY
3 THOR JAMESON, J. JONAH
4 THOR MCCORMICK, BARRY
5 THOR SPIDER-MAN
6 THOR TOKKOTS
7 THOR SOUTHERN, CANDY
8 THOR WASP
9 THOR ANGEL

The duplicates here can be used to calculate the weights of the edges.

In [15]:
print("Total number of links :", len(samp_pophero))
print("Number of duplicates : ", samp_pophero.duplicated().sum())
print("Numer of unique links: ", len(samp_pophero[['hero1', 'hero2']].drop_duplicates()))

Total number of links : 42326
Number of duplicates :  34150
Numer of unique links:  8176

Further prepping sample dataset to consist of link(1) and nolink(0) feature to enable classification modelling.The dataset currently have edges between heroes, and obviously nodes without edges are not represented.

In [16]:
samp_pophero = samp_pophero.drop_duplicates()
In [17]:
herolist1 = list(samp_pophero["hero1"])
herolist2 = list(samp_pophero["hero2"])

set1 = set(herolist1)
set2 = set(herolist2)

herolist_uni = set2 - set1
herolist = list(list(set1) + list(herolist_uni))

print("Number of unique heroes : ", len(herolist))

Number of unique heroes :  3280
In [18]:
herolist[:20]
Out[18]:
['THOR',
 'IRON MAN',
 'WOLVERINE',
 'DR. STRANGE',
 'STORM',
 'SPIDER-MAN',
 'CAPTAIN AMERICA',
 'HULK',
 'KONIEV, MRS.',
 'PUPPY',
 'FAYRELYTE STRONGHEAR',
 'FUGUE',
 'KARMA',
 'TERRAX II',
 'TAGAR',
 'REILLY, MAVIS',
 'ROGERS, CAPT. STEVEN',
 'MAKO',
 'NELSON, CANDACE',
 'LEVIATHAN II']

Important note:
Having 3,280 heroes (nodes) means having 5,377,560 possible links (edges), resulting in massive increment in data that require analysis. Computational requirement increases with each additional node and may become impracticable moving forward.

In [19]:
pairs = list(itertools.combinations(herolist, 2))

df_sample = pd.DataFrame(pairs, columns=['hero1', 'hero2'])
df_sample["link"] = "0"

df_sample
Out[19]:
hero1 hero2 link
0 THOR IRON MAN 0
1 THOR WOLVERINE 0
2 THOR DR. STRANGE 0
3 THOR STORM 0
4 THOR SPIDER-MAN 0
... ... ... ...
5377555 MODULAR MAN HARDERMAN, QUENTIN 0
5377556 MODULAR MAN ARAGORN 0
5377557 DORMA II [ATLANTEAN] HARDERMAN, QUENTIN 0
5377558 DORMA II [ATLANTEAN] ARAGORN 0
5377559 HARDERMAN, QUENTIN ARAGORN 0

5377560 rows × 3 columns

In [20]:
df_sample[20000:20020]
Out[20]:
hero1 hero2 link
20000 CAPTAIN AMERICA RUDOLFO, PRINCE 0
20001 CAPTAIN AMERICA TORK, SGT. 0
20002 CAPTAIN AMERICA ARIZE 0
20003 CAPTAIN AMERICA ELLENTHORPE, MARIAH 0
20004 CAPTAIN AMERICA SHOOTING STAR 0
20005 CAPTAIN AMERICA DARKOTH 0
20006 CAPTAIN AMERICA PETROVITCH, IVAN 0
20007 CAPTAIN AMERICA SPYMASTER 0
20008 CAPTAIN AMERICA PIRANHA 0
20009 CAPTAIN AMERICA AJAX 0
20010 CAPTAIN AMERICA MALICE V 0
20011 CAPTAIN AMERICA LANG, DR. STEVEN 0
20012 CAPTAIN AMERICA NANNY 0
20013 CAPTAIN AMERICA BELASCO 0
20014 CAPTAIN AMERICA YUAN, ANNIE 0
20015 CAPTAIN AMERICA GAMBLE, GRACIE 0
20016 CAPTAIN AMERICA DRAKE, FRANKLIN 0
20017 CAPTAIN AMERICA MOONSTONE II 0
20018 CAPTAIN AMERICA OX III 0
20019 CAPTAIN AMERICA DISTORTER 0
In [21]:
%%time

# define function to apply to each row of df_sample
def check_rows(row):
    if (row['hero1'] in df_sample['hero1'].values) and (row['hero2'] in df_sample['hero2'].values):
        return "1"
    elif (row['hero1'] in df_sample['hero2'].values) and (row['hero2'] in df_sample['hero1'].values):
        return "1"
    else:
        return "0"

# apply function to each row of df_sample
df_sample['link'] = samp_pophero.apply(check_rows, axis=1)
df_sample["link"] = df_sample["link"].fillna(0)
df_sample['link'] = df_sample['link'].astype('int')
df_sample

CPU times: total: 15min 52s
Wall time: 15min 53s
Out[21]:
hero1 hero2 link
0 THOR IRON MAN 1
1 THOR WOLVERINE 1
2 THOR DR. STRANGE 1
3 THOR STORM 1
4 THOR SPIDER-MAN 1
... ... ... ...
5377555 MODULAR MAN HARDERMAN, QUENTIN 0
5377556 MODULAR MAN ARAGORN 0
5377557 DORMA II [ATLANTEAN] HARDERMAN, QUENTIN 0
5377558 DORMA II [ATLANTEAN] ARAGORN 0
5377559 HARDERMAN, QUENTIN ARAGORN 0

5377560 rows × 3 columns

In [22]:
print("Total number of links : ", len(df_sample.loc[df_sample["link"]==1]))

Total number of links :  8176
In [23]:
# Saving sample df to pickle

df_sample.to_pickle("data/df_sample.pkl")

Top

Centrality Features ¶

Using centrality to build features between the nodes to prepare a classifying model to predict the links.
NetworkX Link Prediction

In [24]:
df_sample = pd.read_pickle("data/df_sample.pkl")
df_sample
Out[24]:
hero1 hero2 link
0 THOR IRON MAN 1
1 THOR WOLVERINE 1
2 THOR DR. STRANGE 1
3 THOR STORM 1
4 THOR SPIDER-MAN 1
... ... ... ...
5377555 MODULAR MAN HARDERMAN, QUENTIN 0
5377556 MODULAR MAN ARAGORN 0
5377557 DORMA II [ATLANTEAN] HARDERMAN, QUENTIN 0
5377558 DORMA II [ATLANTEAN] ARAGORN 0
5377559 HARDERMAN, QUENTIN ARAGORN 0

5377560 rows × 3 columns

In [25]:
unique_nodes = list(df_sample['hero1']) + list(df_sample["hero2"])
unique_nodes = list(set(unique_nodes))
len(unique_nodes)
Out[25]:
3280
In [26]:
gx = nx.Graph()
In [27]:
gx.add_nodes_from(unique_nodes)
In [28]:
tuples_list = []
df_link  = df_sample[df_sample['link']==1]
for index, row in df_link.iterrows():
    source = row['hero1']
    target = row['hero2']
    
    tuples_list.append((source, target))
    
tuples_list[:10]
Out[28]:
[('THOR', 'IRON MAN'),
 ('THOR', 'WOLVERINE'),
 ('THOR', 'DR. STRANGE'),
 ('THOR', 'STORM'),
 ('THOR', 'SPIDER-MAN'),
 ('THOR', 'CAPTAIN AMERICA'),
 ('THOR', 'HULK'),
 ('THOR', 'KONIEV, MRS.'),
 ('THOR', 'PUPPY'),
 ('THOR', 'FAYRELYTE STRONGHEAR')]
In [29]:
gx.add_edges_from(tuples_list)

Top

Building the features ¶

In [30]:
%%time
pr_dict = nx.pagerank(gx)

CPU times: total: 31.2 ms
Wall time: 24 ms
In [31]:
%%time 
degree_list = nx.degree(gx)

CPU times: total: 0 ns
Wall time: 0 ns
In [32]:
%%time
degCent = nx.degree_centrality(gx)

CPU times: total: 0 ns
Wall time: 1 ms
In [33]:
%%time 
betCent = nx.betweenness_centrality(gx, normalized=True, endpoints=True, k = 50)

CPU times: total: 625 ms
Wall time: 614 ms
In [34]:
%%time 
hubs_dict, auth_dict = nx.hits(gx)

CPU times: total: 31.2 ms
Wall time: 31.9 ms

C:\Users\alex\anaconda3\lib\site-packages\networkx\algorithms\link_analysis\hits_alg.py:78: FutureWarning: adjacency_matrix will return a scipy.sparse array instead of a matrix in Networkx 3.0.
  A = nx.adjacency_matrix(G, nodelist=list(G), dtype=float)
In [35]:
degree_dict = {}

for x,y  in degree_list  :
    degree_dict[x] = y
    
len(degree_dict)
Out[35]:
3280
In [36]:
%%time  
final_data_list = []
def transform_data(df, add_link = True) :
    
    for index, row in df.iterrows():
        src = row['hero1']
        tgt = row['hero2']
        
        data_list = []
        data_list.append(pr_dict[src])
        data_list.append(hubs_dict[src])
        data_list.append(degCent[src])
        data_list.append(betCent[src])
        data_list.append(pr_dict[tgt])
        data_list.append(hubs_dict[tgt])
        data_list.append(degCent[tgt])
        data_list.append(betCent[tgt])
        if add_link == True :
            data_list.append(row['link'])
        
        final_data_list.append(data_list)        
    if add_link == True :
        col_list = ['SrcPageRank','SrcHubs',
                    'SrcDegCnt','SrcBetCnt',
                    'TgtPageRank','TgtHubs',
                    'TgtDegCnt','TgtBetCnt',
                    'Link']
        df_centrality = pd.DataFrame(final_data_list, columns = col_list)        
    else :
        col_list = ['SrcPageRank','SrcHubs',
                    'SrcDegCnt','SrcBetCnt',
                    'TgtPageRank','TgtHubs',
                    'TgtDegCnt','TgtBetCnt',
                   ]
        df_centrality = pd.DataFrame(final_data_list, columns = col_list)        
    return df_centrality
              

CPU times: total: 0 ns
Wall time: 0 ns
In [37]:
%%time
df_centrality = transform_data(df_sample)        
print(len(final_data_list))        
print(len(df_centrality))

5377560
5377560
CPU times: total: 3min 25s
Wall time: 3min 24s
In [38]:
df_centrality.info()

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 5377560 entries, 0 to 5377559
Data columns (total 9 columns):
 #   Column       Dtype  
---  ------       -----  
 0   SrcPageRank  float64
 1   SrcHubs      float64
 2   SrcDegCnt    float64
 3   SrcBetCnt    float64
 4   TgtPageRank  float64
 5   TgtHubs      float64
 6   TgtDegCnt    float64
 7   TgtBetCnt    float64
 8   Link         int64  
dtypes: float64(8), int64(1)
memory usage: 369.2 MB

Hubs & Auth perfect multicoliinearity. Other features also have very high correlation.

In [39]:
df_centrality.corr()
Out[39]:
SrcPageRank SrcHubs SrcDegCnt SrcBetCnt TgtPageRank TgtHubs TgtDegCnt TgtBetCnt Link
SrcPageRank 1.000000 0.945846 0.999922 0.920453 0.038528 0.011301 0.037856 0.011062 0.447305
SrcHubs 0.945846 1.000000 0.947411 0.889109 0.036665 0.012213 0.036155 0.011028 0.420566
SrcDegCnt 0.999922 0.947411 1.000000 0.921659 0.038480 0.011320 0.037812 0.011063 0.447300
SrcBetCnt 0.920453 0.889109 0.921659 1.000000 0.035161 0.010668 0.034561 0.010066 0.412409
TgtPageRank 0.038528 0.036665 0.038480 0.035161 1.000000 0.526165 0.997913 0.212376 0.035040
TgtHubs 0.011301 0.012213 0.011320 0.010668 0.526165 1.000000 0.577211 0.077373 0.026952
TgtDegCnt 0.037856 0.036155 0.037812 0.034561 0.997913 0.577211 1.000000 0.209274 0.035703
TgtBetCnt 0.011062 0.011028 0.011063 0.010066 0.212376 0.077373 0.209274 1.000000 0.007955
Link 0.447305 0.420566 0.447300 0.412409 0.035040 0.026952 0.035703 0.007955 1.000000
In [40]:
## Saving the centrality dataset

df_centrality.to_pickle("data/df_centrality.pkl")

Top

Building the classification model ¶

In [41]:
df_centrality = pd.read_pickle("data/df_centrality.pkl")
df_centrality
Out[41]:
SrcPageRank SrcHubs SrcDegCnt SrcBetCnt TgtPageRank TgtHubs TgtDegCnt TgtBetCnt Link
0 0.038275 0.006957 0.211345 0.207317 0.033070 0.004224 0.177798 0.091786 1
1 0.038275 0.006957 0.211345 0.207317 0.033583 0.005072 0.185117 0.104626 1
2 0.038275 0.006957 0.211345 0.207317 0.043974 0.006169 0.236962 0.093431 1
3 0.038275 0.006957 0.211345 0.207317 0.026150 0.003629 0.144861 0.037111 1
4 0.038275 0.006957 0.211345 0.207317 0.042895 0.006121 0.232083 0.091216 1
... ... ... ... ... ... ... ... ... ...
5377555 0.000190 0.000298 0.000915 0.000386 0.000144 0.000193 0.000610 0.000316 0
5377556 0.000190 0.000298 0.000915 0.000386 0.000048 -0.000000 0.000000 0.000000 0
5377557 0.000096 0.000088 0.000305 0.000287 0.000144 0.000193 0.000610 0.000316 0
5377558 0.000096 0.000088 0.000305 0.000287 0.000048 -0.000000 0.000000 0.000000 0
5377559 0.000144 0.000193 0.000610 0.000316 0.000048 -0.000000 0.000000 0.000000 0

5377560 rows × 9 columns

In [42]:
X = df_centrality.drop('Link', axis =1)
y = df_centrality['Link']
X_train, X_test, y_train, y_test = train_test_split( X, y, test_size = 0.3, random_state = 42) 
print(X_train.shape)
print(X_test.shape)

(3764292, 8)
(1613268, 8)
In [43]:
%%time
classifier = LGBMClassifier(
#             nthread=4,
            n_estimators=10000,
            learning_rate=0.02,
            num_leaves=34,
            colsample_bytree=0.9497036,
            subsample=0.8715623,
            max_depth=8,
            reg_alpha=0.041545473,
            reg_lambda=0.0735294,
            min_split_gain=0.0222415,
            min_child_weight=39.3259775,
            verbose=-1 )

classifier.fit(X_train, y_train)

y_train_pred = classifier.predict(X_train)
y_test_pred = classifier.predict(X_test) 

CPU times: total: 1h 29min 39s
Wall time: 8min 31s
In [44]:
%%time

train_acc = accuracy_score(y_train,y_train_pred)
test_acc = accuracy_score(y_test,y_test_pred)
print("Accuracy for Gradient Boost model for Train data {:.4f} %".format(train_acc*100))
print("Accuracy for Gradient Boost model for Test data {:.4f} %".format(test_acc*100))

Accuracy for Gradient Boost model for Train data 99.9233 %
Accuracy for Gradient Boost model for Test data 99.9076 %
CPU times: total: 328 ms
Wall time: 273 ms
In [45]:
%%time

print('Balanced accuracy for Gradient Boost Model for Train data {:.4f} %'.format(balanced_accuracy_score(y_train,y_train_pred)*100))
print('Balanced accuracy for Gradient Boost Model for Test data {:.4f} %'.format(balanced_accuracy_score(y_test,y_test_pred)*100))

Balanced accuracy for Gradient Boost Model for Train data 79.8042 %
Balanced accuracy for Gradient Boost Model for Test data 76.3992 %
CPU times: total: 609 ms
Wall time: 585 ms
In [46]:
train_cm = confusion_matrix(y_train,y_train_pred)
test_cm = confusion_matrix(y_test,y_test_pred)
print("Confusion Matrix for Gradient Boost model for Train data :\n" , train_cm)
print("Confusion Matrix for Gradient Boost model for Test data :\n" , test_cm)

Confusion Matrix for Gradient Boost model for Train data :
 [[3757958     551]
 [   2335    3448]]
Confusion Matrix for Gradient Boost model for Test data :
 [[1610514     361]
 [   1129    1264]]
In [47]:
print('Classification Report on Train Data\n',classification_report(y_train,y_train_pred))
print('Confusion Matrix on Train Data\n',confusion_matrix(y_train, y_train_pred))
print('\nClassification Report on Test Data\n',classification_report(y_test,y_test_pred))
print('Confusion Matrix on Test Data\n',confusion_matrix(y_test, y_test_pred))

Classification Report on Train Data
               precision    recall  f1-score   support

           0       1.00      1.00      1.00   3758509
           1       0.86      0.60      0.70      5783

    accuracy                           1.00   3764292
   macro avg       0.93      0.80      0.85   3764292
weighted avg       1.00      1.00      1.00   3764292

Confusion Matrix on Train Data
 [[3757958     551]
 [   2335    3448]]

Classification Report on Test Data
               precision    recall  f1-score   support

           0       1.00      1.00      1.00   1610875
           1       0.78      0.53      0.63      2393

    accuracy                           1.00   1613268
   macro avg       0.89      0.76      0.81   1613268
weighted avg       1.00      1.00      1.00   1613268

Confusion Matrix on Test Data
 [[1610514     361]
 [   1129    1264]]

Top

In-Built Algorithm ¶

Jaccard Coefficient, Adamic-Adar and Prederential Attachment
Graph Learning

In [48]:
samp_pophero.head()
Out[48]:
hero1 hero2
0 THOR GRANT, GLORIA GLORY
1 THOR FAIRMONT, HANNAH
2 THOR PARKER, MAY
3 THOR JAMESON, J. JONAH
4 THOR MCCORMICK, BARRY
In [49]:
samp_pophero.reset_index(inplace=True, drop=True)
samp_pophero.info
Out[49]:
<bound method DataFrame.info of             hero1                hero2
0            THOR  GRANT, GLORIA GLORY
1            THOR     FAIRMONT, HANNAH
2            THOR          PARKER, MAY
3            THOR    JAMESON, J. JONAH
4            THOR     MCCORMICK, BARRY
...           ...                  ...
8171  DR. STRANGE             CALLISTO
8172  DR. STRANGE                MAGMA
8173  DR. STRANGE               SUNDER
8174  DR. STRANGE           KULAN GATH
8175  DR. STRANGE  GYRICH, HENRY PETER

[8176 rows x 2 columns]>
In [50]:
Gb = nx.from_pandas_edgelist(samp_pophero, 'hero1', 'hero2')
In [51]:
n = Gb.number_of_nodes()
m = Gb.number_of_edges()
print("Number of nodes : %d" % n)
print("Number of edges : %d" % m)
print("Number of connected components : %d" % nx.number_connected_components(Gb))

Number of nodes : 3280
Number of edges : 8148
Number of connected components : 1
In [52]:
plt.figure(figsize=(18,18))
nx.draw(Gb)
plt.gca().collections[0].set_edgecolor("#000000")
In [53]:
# Remove 20% of the edges
random.seed(42)
proportion_edges = 0.2
edge_subset = random.sample(Gb.edges(), int(proportion_edges * Gb.number_of_edges()))

# Create a copy of the graph and remove the edges
Gb_train = Gb.copy()
Gb_train.remove_edges_from(edge_subset)

In [54]:
plt.figure(figsize=(18,18))
nx.draw(Gb_train)
plt.gca().collections[0].set_edgecolor("#000000") # set node border color to black
In [55]:
edge_subset_size = len(list(edge_subset))
print("Number of edges deleted : %d" % edge_subset_size)
print("Number of edges remaining : %d" % (m - edge_subset_size))

Number of edges deleted : 1629
Number of edges remaining : 6519

Top

Jaccard Coefficient ¶

In [56]:
%%time
# Make prediction using Jaccard Coefficient
pred_jaccard = list(nx.jaccard_coefficient(Gb_train))
score_jaccard, label_jaccard = zip(*[(s, (u,v) in edge_subset) for (u,v,s) in pred_jaccard])

fpr_jaccard, tpr_jaccard, _ = roc_curve(label_jaccard, score_jaccard)
auc_jaccard = round(roc_auc_score(label_jaccard, score_jaccard), 4)

CPU times: total: 3min 10s
Wall time: 3min 10s
In [57]:
pred_jaccard
Out[57]:
[('KONIEV, MRS.', 'PUPPY', 0.0),
 ('KONIEV, MRS.', 'FAYRELYTE STRONGHEAR', 0.2),
 ('KONIEV, MRS.', 'FUGUE', 0.0),
 ('KONIEV, MRS.', 'KARMA', 0.0),
 ('KONIEV, MRS.', 'TERRAX II', 0.0),
 ('KONIEV, MRS.', 'TAGAR', 0.0),
 ('KONIEV, MRS.', 'REILLY, MAVIS', 0.0),
 ('KONIEV, MRS.', 'ROGERS, CAPT. STEVEN', 0.0),
 ('KONIEV, MRS.', 'MAKO', 0.0),
 ('KONIEV, MRS.', 'NELSON, CANDACE', 0.0),
 ('KONIEV, MRS.', 'LEVIATHAN II', 0.0),
 ('KONIEV, MRS.', 'MYSTERIA', 0.0),
 ('KONIEV, MRS.', 'ROSS, EVERETT KENNET', 0.2),
 ('KONIEV, MRS.', 'BUTTERFLY', 0.0),
 ('KONIEV, MRS.', 'JEFFERSON, DR.', 0.0),
 ('KONIEV, MRS.', 'SMITH, JOE', 0.0),
 ('KONIEV, MRS.', 'MAXIMUS [INHUMAN]', 0.0),
 ('KONIEV, MRS.', 'ROCKETLAUNCHER', 0.0),
 ('KONIEV, MRS.', 'ASP II', 0.6666666666666666),
 ('KONIEV, MRS.', 'BUNYAN, PAUL', 0.0),
 ('KONIEV, MRS.', 'AUTHORITY', 0.0),
 ('KONIEV, MRS.', 'FU MANCHU', 0.0),
 ('KONIEV, MRS.', 'FLAME II', 0.0),
 ('KONIEV, MRS.', 'TAYLOR, JENNA', 0.0),
 ('KONIEV, MRS.', 'TECHNOSPIKE', 0.0),
 ('KONIEV, MRS.', 'EGO-PRIME', 0.0),
 ('KONIEV, MRS.', 'AMPHIBIUS', 0.6666666666666666),
 ('KONIEV, MRS.', 'SKUNGE', 0.0),
 ('KONIEV, MRS.', 'MAGMA', 0.14285714285714285),
 ('KONIEV, MRS.', 'SPEEDBALL II', 0.0),
 ('KONIEV, MRS.', 'RAKESH', 0.0),
 ('KONIEV, MRS.', 'DOMINUS II', 0.0),
 ('KONIEV, MRS.', 'HALF-FACE', 0.0),
 ('KONIEV, MRS.', 'OSBORN, NORMAN', 0.0),
 ('KONIEV, MRS.', 'ERWIN, MORLEY', 0.0),
 ('KONIEV, MRS.', 'MACKENZIE, AL', 0.3333333333333333),
 ('KONIEV, MRS.', 'MCCORMICK, BARRY', 0.0),
 ('KONIEV, MRS.', 'CREEP', 0.0),
 ('KONIEV, MRS.', 'LUPO', 0.0),
 ('KONIEV, MRS.', 'SWORDSMAN III', 0.0),
 ('KONIEV, MRS.', 'GREEN GOBLIN V', 0.0),
 ('KONIEV, MRS.', 'HULK IV', 0.0),
 ('KONIEV, MRS.', 'HARPOON', 0.6666666666666666),
 ('KONIEV, MRS.', 'BANNON, LANCE', 0.0),
 ('KONIEV, MRS.', 'CHALIZ, SHANAZ RAYNA', 0.0),
 ('KONIEV, MRS.', 'TORK, LT.', 0.0),
 ('KONIEV, MRS.', 'POWERHOUSE II', 0.0),
 ('KONIEV, MRS.', 'JACKSON, CURTISS', 0.0),
 ('KONIEV, MRS.', 'FERAL', 0.3333333333333333),
 ('KONIEV, MRS.', 'JOYBOY', 0.0),
 ('KONIEV, MRS.', 'THALLIUM', 0.0),
 ('KONIEV, MRS.', 'YANDROTH', 0.0),
 ('KONIEV, MRS.', 'LOCKLIN, BEN', 0.5),
 ('KONIEV, MRS.', 'SZARDOS, MARGALI', 1.0),
 ('KONIEV, MRS.', 'OSBORN, LIZ ALLAN', 0.0),
 ('KONIEV, MRS.', 'COLOSSUS 2013', 0.5),
 ('KONIEV, MRS.', 'HOPKINS, STEVE', 0.0),
 ('KONIEV, MRS.', 'SHIVER MAN', 0.5),
 ('KONIEV, MRS.', 'WILD ONE', 0.0),
 ('KONIEV, MRS.', 'POWELL', 0.0),
 ('KONIEV, MRS.', 'MONGOOSE', 0.0),
 ('KONIEV, MRS.', 'TINKERER', 0.0),
 ('KONIEV, MRS.', 'CARPENTER, RACHEL', 0.0),
 ('KONIEV, MRS.', 'ROMMEL, EDWIN ROMULU', 0.3333333333333333),
 ('KONIEV, MRS.', 'CALCULUS', 0.0),
 ('KONIEV, MRS.', 'RAVEN, LT. JACOB', 0.0),
 ('KONIEV, MRS.', 'BELLADONNA', 0.0),
 ('KONIEV, MRS.', 'GRASP', 0.0),
 ('KONIEV, MRS.', 'MISSING LINK', 0.0),
 ('KONIEV, MRS.', 'RAPTOR II', 0.5),
 ('KONIEV, MRS.', 'NUKE', 0.0),
 ('KONIEV, MRS.', 'GRIMM, BEN | TIMESLI', 0.0),
 ('KONIEV, MRS.', 'DRAKE, CARLTON', 0.0),
 ('KONIEV, MRS.', 'GORN', 0.0),
 ('KONIEV, MRS.', 'BROWN, ABE', 0.0),
 ('KONIEV, MRS.', 'PROFESSOR X', 0.2857142857142857),
 ('KONIEV, MRS.', 'ANNALEE', 0.5),
 ('KONIEV, MRS.', 'RADIOACTIVE MAN', 0.0),
 ('KONIEV, MRS.', 'JUSTICE PEACE', 0.0),
 ('KONIEV, MRS.', 'HUNCH', 0.0),
 ('KONIEV, MRS.', 'MCKEEVER, TINY', 0.0),
 ('KONIEV, MRS.', 'HAG', 0.0),
 ('KONIEV, MRS.', 'MAGUS', 0.2857142857142857),
 ('KONIEV, MRS.', 'ATTRACTIVE LAD', 0.0),
 ('KONIEV, MRS.', 'HULK DOPPELGANGER II', 0.4),
 ('KONIEV, MRS.', 'DUFFY, SGT. MIKE', 0.0),
 ('KONIEV, MRS.', 'CYBERMANCER', 0.0),
 ('KONIEV, MRS.', 'SCARLET WITCH', 0.3333333333333333),
 ('KONIEV, MRS.', 'ERBIUM', 0.0),
 ('KONIEV, MRS.', 'GREGORY, TODD', 0.0),
 ('KONIEV, MRS.', 'NUGENT, ALICE', 0.0),
 ('KONIEV, MRS.', 'MIKHLO', 0.14285714285714285),
 ('KONIEV, MRS.', 'SNAKE-EYES', 0.0),
 ('KONIEV, MRS.', 'MADAME MASQUE IV', 0.0),
 ('KONIEV, MRS.', 'FORTUNE, DOMINIC', 0.0),
 ('KONIEV, MRS.', 'ROBERTSON, MARTHA', 0.3333333333333333),
 ('KONIEV, MRS.', 'LAVEAU, MARIE', 0.0),
 ('KONIEV, MRS.', 'SPOT', 0.0),
 ('KONIEV, MRS.', 'MESMERO', 0.6666666666666666),
 ('KONIEV, MRS.', 'MOLE MAN', 0.25),
 ('KONIEV, MRS.', 'MEACHUM, JOY', 0.3333333333333333),
 ('KONIEV, MRS.', 'BLACK MAMBA', 0.3333333333333333),
 ('KONIEV, MRS.', 'MONTGOMERY', 0.0),
 ('KONIEV, MRS.', 'SKYHAWK', 0.0),
 ('KONIEV, MRS.', 'LEONUS', 0.0),
 ('KONIEV, MRS.', 'WHIZZER III', 0.0),
 ('KONIEV, MRS.', 'TANA NILE', 0.0),
 ('KONIEV, MRS.', 'STEELE, BOBBY', 0.0),
 ('KONIEV, MRS.', 'HERCULES DOPPELGANGE', 0.5),
 ('KONIEV, MRS.', 'MISS ITCH', 0.0),
 ('KONIEV, MRS.', 'BEYNON, DR. BRADLEY', 0.5),
 ('KONIEV, MRS.', 'PIPELINE', 0.0),
 ('KONIEV, MRS.', 'SUB-MARINER', 0.125),
 ('KONIEV, MRS.', 'SCORCHER', 0.0),
 ('KONIEV, MRS.', 'WATANABE, YUJI', 0.5),
 ('KONIEV, MRS.', 'RAA', 0.0),
 ('KONIEV, MRS.', 'COURTNEY, DELPHINE', 0.0),
 ('KONIEV, MRS.', 'ATALANTA', 0.0),
 ('KONIEV, MRS.', 'AGED GENGHIS', 0.0),
 ('KONIEV, MRS.', 'REINMAN, PAULA', 1.0),
 ('KONIEV, MRS.', 'SATAN', 0.0),
 ('KONIEV, MRS.', 'FOXBAT', 0.0),
 ('KONIEV, MRS.', 'JUSTICE II', 0.4),
 ('KONIEV, MRS.', 'WINGFOOT, WYNONA', 0.0),
 ('KONIEV, MRS.', 'GAMELIN, JOHN T J.T.', 0.0),
 ('KONIEV, MRS.', 'TITAN, TRISH', 0.0),
 ('KONIEV, MRS.', 'DR. PHOBOS', 0.0),
 ('KONIEV, MRS.', 'HINKLEY', 0.0),
 ('KONIEV, MRS.', 'KAISHEK, HAAN', 0.5),
 ('KONIEV, MRS.', 'MILLER, STACI MURPHY', 0.5),
 ('KONIEV, MRS.', 'RAZORFIST III', 0.5),
 ('KONIEV, MRS.', 'SWORDSMAN', 0.14285714285714285),
 ('KONIEV, MRS.', 'BOND', 0.5),
 ('KONIEV, MRS.', 'DORREK II', 0.0),
 ('KONIEV, MRS.', 'TAE', 0.0),
 ('KONIEV, MRS.', 'DEATHURGE', 0.2),
 ('KONIEV, MRS.', 'LUMP', 0.0),
 ('KONIEV, MRS.', 'HOGARTH, JERYN', 0.0),
 ('KONIEV, MRS.', 'DOLMAN', 0.0),
 ('KONIEV, MRS.', 'SPARK', 0.0),
 ('KONIEV, MRS.', 'AJAK', 0.3333333333333333),
 ('KONIEV, MRS.', 'LEO', 0.0),
 ('KONIEV, MRS.', 'STARK, MORGAN', 0.0),
 ('KONIEV, MRS.', 'POWELL, AMY', 0.0),
 ('KONIEV, MRS.', 'ZEUS', 0.0),
 ('KONIEV, MRS.', 'FORGE', 0.3333333333333333),
 ('KONIEV, MRS.', 'HIRO', 0.5),
 ('KONIEV, MRS.', 'PHOENIX III', 0.25),
 ('KONIEV, MRS.', 'DOCKERY, RUPERT', 0.0),
 ('KONIEV, MRS.', 'CAPTAIN WINGS', 0.0),
 ('KONIEV, MRS.', 'PUENTES, DONNA MARIA', 0.3333333333333333),
 ('KONIEV, MRS.', 'KELLEHER, MARY', 0.0),
 ('KONIEV, MRS.', 'TYNDAR', 0.0),
 ('KONIEV, MRS.', 'PSI-LORD', 0.0),
 ('KONIEV, MRS.', 'CABE, BETHANY', 0.0),
 ('KONIEV, MRS.', 'HAYWIRE', 0.0),
 ('KONIEV, MRS.', 'HELLEN', 0.0),
 ('KONIEV, MRS.', 'HOUND', 0.5),
 ('KONIEV, MRS.', 'MEPHISTA', 0.0),
 ('KONIEV, MRS.', 'MAN-BEAST', 0.0),
 ('KONIEV, MRS.', 'PORCAL', 0.0),
 ('KONIEV, MRS.', 'HUMAN ROBOT', 0.0),
 ('KONIEV, MRS.', 'FLINN, LANCELOT LANC', 0.0),
 ('KONIEV, MRS.', 'KREA, BRUNO', 0.5),
 ('KONIEV, MRS.', 'SPEEDBALL', 0.4),
 ('KONIEV, MRS.', 'GRIM HUNTER', 0.0),
 ('KONIEV, MRS.', 'ISIS', 0.0),
 ('KONIEV, MRS.', 'STILT-MAN', 0.0),
 ('KONIEV, MRS.', 'CRIMSON DYNAMO II', 0.0),
 ('KONIEV, MRS.', 'REPTYL', 0.0),
 ('KONIEV, MRS.', 'MIRAGE', 0.0),
 ('KONIEV, MRS.', 'RICHARDS, NATHANIEL', 0.0),
 ('KONIEV, MRS.', 'GUNDERSON, COL. ROLA', 0.0),
 ('KONIEV, MRS.', 'X-RAY', 0.14285714285714285),
 ('KONIEV, MRS.', 'JANUS', 0.0),
 ('KONIEV, MRS.', 'PREYY', 0.0),
 ('KONIEV, MRS.', 'HURRICANE III', 1.0),
 ('KONIEV, MRS.', 'STARK, HOWARD', 0.0),
 ('KONIEV, MRS.', 'GERARD, TERENCE', 0.0),
 ('KONIEV, MRS.', 'FREEMAN, DUANE JEROM', 0.0),
 ('KONIEV, MRS.', 'ANT-MAN II', 0.14285714285714285),
 ('KONIEV, MRS.', 'CARGIL', 0.6666666666666666),
 ('KONIEV, MRS.', 'TYPHOID MARY', 0.25),
 ('KONIEV, MRS.', 'RED SKULL II', 0.0),
 ('KONIEV, MRS.', 'MASTER PANDEMONIUM', 0.0),
 ('KONIEV, MRS.', 'KAINE', 0.0),
 ('KONIEV, MRS.', 'JENSKOT', 0.5),
 ('KONIEV, MRS.', 'BERGIER, REBECCA', 0.0),
 ('KONIEV, MRS.', 'GROSS, IRA', 0.0),
 ('KONIEV, MRS.', 'STEVENS, MILO', 0.0),
 ('KONIEV, MRS.', 'FATALE', 0.5),
 ('KONIEV, MRS.', 'CYBER', 1.0),
 ('KONIEV, MRS.', 'ENGLEHART, STEVE', 0.0),
 ('KONIEV, MRS.', 'MUTO, JUNZO', 0.25),
 ('KONIEV, MRS.', 'MOON KNIGHT DOPPELGA', 0.6666666666666666),
 ('KONIEV, MRS.', 'DAZZLER II', 0.16666666666666666),
 ('KONIEV, MRS.', 'EXPEDITER II', 0.6666666666666666),
 ('KONIEV, MRS.', 'LIFEFORCE', 0.5),
 ('KONIEV, MRS.', 'GARDENER', 0.0),
 ('KONIEV, MRS.', 'BEAUTIFUL DREAMER', 0.5),
 ('KONIEV, MRS.', 'MAX II', 0.0),
 ('KONIEV, MRS.', 'FOUNDRY', 0.0),
 ('KONIEV, MRS.', 'HALLER, GABRIELLE', 0.5),
 ('KONIEV, MRS.', 'RASPUTIN, NIKOLAI', 0.5),
 ('KONIEV, MRS.', 'HAWKEYE DOPPELGANGER', 0.0),
 ('KONIEV, MRS.', 'COUNTERWEIGHT II', 0.25),
 ('KONIEV, MRS.', 'TARANTULA II', 0.0),
 ('KONIEV, MRS.', 'REBEL', 0.0),
 ('KONIEV, MRS.', 'GROWING MAN', 0.0),
 ('KONIEV, MRS.', 'CRIMSON DYNAMO V', 0.4),
 ('KONIEV, MRS.', 'BEAST', 0.3333333333333333),
 ('KONIEV, MRS.', 'JOCASTA', 0.16666666666666666),
 ('KONIEV, MRS.', 'EXITAR', 0.0),
 ('KONIEV, MRS.', 'BONES', 0.0),
 ('KONIEV, MRS.', 'SHAW, MILDRED', 0.0),
 ('KONIEV, MRS.', 'CARGO', 0.0),
 ('KONIEV, MRS.', 'THERAK', 0.0),
 ('KONIEV, MRS.', 'BUTCH', 0.5),
 ('KONIEV, MRS.', 'PRAZNIKI, SYLVIA', 0.0),
 ('KONIEV, MRS.', 'WASP', 0.2857142857142857),
 ('KONIEV, MRS.', 'FANG II', 0.0),
 ('KONIEV, MRS.', 'MONTGOMERY, SIR BERN', 0.0),
 ('KONIEV, MRS.', 'CLOWN', 0.0),
 ('KONIEV, MRS.', 'FLAG-SMASHER', 0.0),
 ('KONIEV, MRS.', 'ANDERSON, CHARLES', 0.0),
 ('KONIEV, MRS.', "R'TEE", 0.5),
 ('KONIEV, MRS.', 'DRAGONESS', 0.5),
 ('KONIEV, MRS.', 'GRIFFIN II', 0.0),
 ('KONIEV, MRS.', 'DEVILLE, JENNIFER CR', 0.0),
 ('KONIEV, MRS.', 'DIAMONDBACK II', 0.3333333333333333),
 ('KONIEV, MRS.', 'FAMINE', 0.5),
 ('KONIEV, MRS.', 'MISS MASS', 0.3333333333333333),
 ('KONIEV, MRS.', 'CREED', 0.0),
 ('KONIEV, MRS.', 'FREDERICKS, GENERAL', 1.0),
 ('KONIEV, MRS.', 'DIXON, GENERAL', 0.0),
 ('KONIEV, MRS.', 'RADIUS', 1.0),
 ('KONIEV, MRS.', 'SOPHIE', 0.0),
 ('KONIEV, MRS.', 'BERNSTEIN, DR. NOAH', 0.0),
 ('KONIEV, MRS.', 'BARON BLOOD II', 0.0),
 ('KONIEV, MRS.', 'URICH, BEN', 0.0),
 ('KONIEV, MRS.', 'KLAW', 0.16666666666666666),
 ('KONIEV, MRS.', 'MOON WOLF', 0.5),
 ('KONIEV, MRS.', 'GREY GARGOYLE', 0.0),
 ('KONIEV, MRS.', 'DR. DRUID', 0.2857142857142857),
 ('KONIEV, MRS.', 'KILLER SHRIKE', 0.0),
 ('KONIEV, MRS.', 'SHINSKI, PROFESSOR W', 0.0),
 ('KONIEV, MRS.', 'MELLACE, LUCY SANTIN', 0.5),
 ('KONIEV, MRS.', 'WILD CHILD', 0.2857142857142857),
 ('KONIEV, MRS.', 'MR. JIP', 0.5),
 ('KONIEV, MRS.', 'MASTERS, DR. AUGUST', 0.0),
 ('KONIEV, MRS.', 'MENKEN, DONALD L.', 0.0),
 ('KONIEV, MRS.', 'MIDNIGHT III', 0.0),
 ('KONIEV, MRS.', 'RAMBO', 0.0),
 ('KONIEV, MRS.', 'HUMBUG', 0.0),
 ('KONIEV, MRS.', 'BENDIX, DR. ALBERT', 0.0),
 ('KONIEV, MRS.', 'DABOIA, RUSSELL', 0.0),
 ('KONIEV, MRS.', 'GILMORE, BERT', 0.0),
 ('KONIEV, MRS.', 'MASTER MENACE', 0.0),
 ('KONIEV, MRS.', 'RIGER', 0.0),
 ('KONIEV, MRS.', 'PARKER, MAY III', 0.0),
 ('KONIEV, MRS.', 'KENT, CLARK', 0.5),
 ('KONIEV, MRS.', 'COY, GENERAL NGUYEN ', 0.0),
 ('KONIEV, MRS.', 'HELL-EYES', 0.0),
 ('KONIEV, MRS.', 'DUGAN, TIMOTHY ALOYI', 0.0),
 ('KONIEV, MRS.', 'LIPSCOMB, DR. ANGELA', 0.0),
 ('KONIEV, MRS.', 'RIOT-ACT', 0.0),
 ('KONIEV, MRS.', 'KHANDRUVITCH, DR. VA', 0.0),
 ('KONIEV, MRS.', 'PRYDE, CARMEN', 0.5),
 ('KONIEV, MRS.', 'NANNY II', 0.3333333333333333),
 ('KONIEV, MRS.', 'GREEN GOBLIN IV', 0.0),
 ('KONIEV, MRS.', 'STONE-FACE', 0.0),
 ('KONIEV, MRS.', 'NIGHT THRASHER', 0.0),
 ('KONIEV, MRS.', 'WEASON, SENATOR', 0.0),
 ('KONIEV, MRS.', 'CYRA [ASGARDIAN]', 0.0),
 ('KONIEV, MRS.', 'BLACK KNIGHT IV DOPP', 0.2),
 ('KONIEV, MRS.', 'MANIPULATOR', 0.0),
 ('KONIEV, MRS.', 'MAELSTROM', 0.0),
 ('KONIEV, MRS.', 'LANE, LOIS', 0.6666666666666666),
 ('KONIEV, MRS.', 'GESTALT', 0.0),
 ('KONIEV, MRS.', 'STARSHINE II', 0.2),
 ('KONIEV, MRS.', 'MENTOR II', 0.6666666666666666),
 ('KONIEV, MRS.', 'GREASE', 0.0),
 ('KONIEV, MRS.', 'FLYNN', 0.0),
 ('KONIEV, MRS.', 'HAUPTMAN', 0.0),
 ('KONIEV, MRS.', 'CROSSBONES', 0.0),
 ('KONIEV, MRS.', 'SLEEPWALKER', 0.0),
 ('KONIEV, MRS.', 'EPHESUS', 0.0),
 ('KONIEV, MRS.', 'MAN-KILLER', 0.0),
 ('KONIEV, MRS.', 'ITZAMNA', 0.16666666666666666),
 ('KONIEV, MRS.', 'HUMAN TORCH ANDROID', 0.2),
 ('KONIEV, MRS.', 'UMAR', 0.0),
 ('KONIEV, MRS.', 'LIBRA III', 0.0),
 ('KONIEV, MRS.', 'SCARECROW', 0.0),
 ('KONIEV, MRS.', 'GAZA', 0.4),
 ('KONIEV, MRS.', 'TREVANE, DET. CONNOR', 0.0),
 ('KONIEV, MRS.', 'CROWLEY, SEN.', 0.0),
 ('KONIEV, MRS.', 'MICROCHIP', 0.0),
 ('KONIEV, MRS.', 'INFOMORPH', 0.0),
 ('KONIEV, MRS.', 'GOG', 0.0),
 ('KONIEV, MRS.', 'CYCLOPS II', 0.5),
 ('KONIEV, MRS.', 'SPIDER-MAN', 0.001943634596695821),
 ('KONIEV, MRS.', 'STONE, CHANTAL', 0.0),
 ('KONIEV, MRS.', 'MARKS, DR. SHIELA', 0.0),
 ('KONIEV, MRS.', 'CYNAS', 0.0),
 ('KONIEV, MRS.', 'MAD DOG', 0.0),
 ('KONIEV, MRS.', 'PENNSU', 0.0),
 ('KONIEV, MRS.', 'ANGEL III', 0.0),
 ('KONIEV, MRS.', 'BENWAY, DR.', 0.0),
 ('KONIEV, MRS.', 'NEWMARK, LOTUS', 0.16666666666666666),
 ('KONIEV, MRS.', 'STEIN, EMIL', 0.0),
 ('KONIEV, MRS.', 'YELLOW CLAW', 0.0),
 ('KONIEV, MRS.', 'CALIFORNIUM', 0.0),
 ('KONIEV, MRS.', 'SSSITH', 0.0),
 ('KONIEV, MRS.', 'BOGATYR', 0.5),
 ('KONIEV, MRS.', 'BOWMAN, MAJ.', 0.0),
 ('KONIEV, MRS.', 'MARGOYLE', 0.0),
 ('KONIEV, MRS.', 'CONNORS, MARTHA', 0.0),
 ('KONIEV, MRS.', 'CAPTAIN MARVEL II', 0.25),
 ('KONIEV, MRS.', 'FOOLKILLER III', 0.0),
 ('KONIEV, MRS.', 'LOGA', 0.0),
 ('KONIEV, MRS.', 'AUSTIN, TERRY', 1.0),
 ('KONIEV, MRS.', 'UNCLE SAM', 0.0),
 ('KONIEV, MRS.', 'GRON', 0.0),
 ('KONIEV, MRS.', 'CHAOS', 1.0),
 ('KONIEV, MRS.', 'FURY, COL. NICHOLAS ', 0.16666666666666666),
 ('KONIEV, MRS.', 'ODDBALL', 0.0),
 ('KONIEV, MRS.', 'DIBBS, LIONEL', 0.0),
 ('KONIEV, MRS.', 'LIGHTSPEED', 0.0),
 ('KONIEV, MRS.', 'CRIME-MASTER', 0.0),
 ('KONIEV, MRS.', 'FRIEDLANDER, SHARON', 1.0),
 ('KONIEV, MRS.', 'CAPTAIN MARVEL', 0.3333333333333333),
 ('KONIEV, MRS.', 'NELIT', 0.0),
 ('KONIEV, MRS.', 'BLESSING, MORGANA', 0.0),
 ('KONIEV, MRS.', 'ABRAXAS', 0.0),
 ('KONIEV, MRS.', 'MACRAE, TRUTH', 0.0),
 ('KONIEV, MRS.', 'RENKO, DR. IRENO', 0.0),
 ('KONIEV, MRS.', 'BAXTER, ABIGAIL', 0.0),
 ('KONIEV, MRS.', 'MAVERICK', 1.0),
 ('KONIEV, MRS.', 'TOLLIVER, SHERIFF JO', 0.0),
 ('KONIEV, MRS.', 'RUDOLFO, PRINCE', 0.0),
 ('KONIEV, MRS.', 'TORK, SGT.', 0.0),
 ('KONIEV, MRS.', 'ARIZE', 0.5),
 ('KONIEV, MRS.', 'ELLENTHORPE, MARIAH', 0.5),
 ('KONIEV, MRS.', 'SHOOTING STAR', 0.0),
 ('KONIEV, MRS.', 'DARKOTH', 0.0),
 ('KONIEV, MRS.', 'HULK', 0.0031201248049922),
 ('KONIEV, MRS.', 'PETROVITCH, IVAN', 0.0),
 ('KONIEV, MRS.', 'SPYMASTER', 0.0),
 ('KONIEV, MRS.', 'PIRANHA', 0.0),
 ('KONIEV, MRS.', 'AJAX', 0.0),
 ('KONIEV, MRS.', 'MALICE V', 0.2),
 ('KONIEV, MRS.', 'LANG, DR. STEVEN', 1.0),
 ('KONIEV, MRS.', 'NANNY', 0.0),
 ('KONIEV, MRS.', 'BELASCO', 0.5),
 ('KONIEV, MRS.', 'YUAN, ANNIE', 0.0),
 ('KONIEV, MRS.', 'GAMBLE, GRACIE', 0.0),
 ('KONIEV, MRS.', 'DRAKE, FRANKLIN', 0.0),
 ('KONIEV, MRS.', 'MOONSTONE II', 0.0),
 ('KONIEV, MRS.', 'OX III', 0.0),
 ('KONIEV, MRS.', 'DISTORTER', 0.0),
 ('KONIEV, MRS.', 'CLARK, SARAH', 0.5),
 ('KONIEV, MRS.', 'WEISS, KONRAD KONNIE', 0.5),
 ('KONIEV, MRS.', 'BARON BRIMSTONE', 0.0),
 ('KONIEV, MRS.', 'DUNER, MAJOR', 0.0),
 ('KONIEV, MRS.', 'STACY, GWEN (CLONE)', 0.0),
 ('KONIEV, MRS.', 'COOPER, ROGER CAPS', 0.0),
 ('KONIEV, MRS.', 'BERNARD THE POET', 0.5),
 ('KONIEV, MRS.', 'MADDICKS, ARTHUR ART', 0.2),
 ('KONIEV, MRS.', 'YETI', 0.0),
 ('KONIEV, MRS.', 'FIRE EATER', 0.0),
 ('KONIEV, MRS.', 'LEBEAU, JEAN-LUC', 0.0),
 ('KONIEV, MRS.', 'RAZA LONGKNIFE', 0.3333333333333333),
 ('KONIEV, MRS.', 'NOVA II', 0.14285714285714285),
 ('KONIEV, MRS.', 'TRIATHLON', 0.2),
 ('KONIEV, MRS.', 'CERBERUS', 0.0),
 ('KONIEV, MRS.', 'PUPPET MASTER', 0.25),
 ('KONIEV, MRS.', 'SAVANNAH, LOU-ANN', 0.0),
 ('KONIEV, MRS.', 'GRANT, SILAS', 0.0),
 ('KONIEV, MRS.', 'STRANGER', 0.0),
 ('KONIEV, MRS.', 'PENANCE', 1.0),
 ('KONIEV, MRS.', 'NORTH, RICKY', 0.0),
 ('KONIEV, MRS.', 'SILHOUETTE', 0.2857142857142857),
 ('KONIEV, MRS.', 'AVRIL, SALLY', 0.0),
 ('KONIEV, MRS.', 'ASHEMA [CELESTIAL]', 0.0),
 ('KONIEV, MRS.', 'DR. NEMESIS', 0.0),
 ('KONIEV, MRS.', 'RAYMOND, ANN', 0.0),
 ('KONIEV, MRS.', 'TOKKOTS', 0.0),
 ('KONIEV, MRS.', 'DEVIL-SLAYER', 0.14285714285714285),
 ('KONIEV, MRS.', 'LLYRA', 0.2),
 ('KONIEV, MRS.', 'TOLLIVER, MELBA', 0.0),
 ('KONIEV, MRS.', 'CORBEAU, DR. PETER', 0.14285714285714285),
 ('KONIEV, MRS.', 'MALTESE', 0.0),
 ('KONIEV, MRS.', 'FORRESTER, CAPT. ALE', 1.0),
 ('KONIEV, MRS.', 'TORPEDO III', 0.14285714285714285),
 ('KONIEV, MRS.', 'GAUNT', 0.0),
 ('KONIEV, MRS.', 'DAN-FARR', 0.5),
 ('KONIEV, MRS.', 'D.K.', 0.0),
 ('KONIEV, MRS.', 'ZAPPER, DAN', 0.0),
 ('KONIEV, MRS.', 'GREEN, JAMES', 0.0),
 ('KONIEV, MRS.', 'MERCURY III', 1.0),
 ('KONIEV, MRS.', 'ARCADE', 0.5),
 ('KONIEV, MRS.', 'REYES, DR. CECELIA', 0.4),
 ('KONIEV, MRS.', 'BLACK WIDOW DOPPELGA', 0.16666666666666666),
 ('KONIEV, MRS.', 'KAMINARI', 0.0),
 ('KONIEV, MRS.', 'SOCRATES SOCK', 0.0),
 ('KONIEV, MRS.', 'PARKER, MARY', 0.0),
 ('KONIEV, MRS.', 'CHESBRO', 0.0),
 ('KONIEV, MRS.', 'MASTERBLASTER', 0.0),
 ('KONIEV, MRS.', 'AVRIL, YVETTE', 0.0),
 ('KONIEV, MRS.', 'GRAVEMOSS', 0.5),
 ('KONIEV, MRS.', 'STACY, CAPT. GEORGE', 0.0),
 ('KONIEV, MRS.', 'PALERMO, VICTOR', 1.0),
 ('KONIEV, MRS.', 'PHANTOM RIDER II', 0.0),
 ('KONIEV, MRS.', 'SYBYL DORN', 0.0),
 ('KONIEV, MRS.', 'PEARSON, AUBREY', 0.0),
 ('KONIEV, MRS.', 'OSBORN, NORMIE', 0.0),
 ('KONIEV, MRS.', 'WAR MACHINE II', 0.0),
 ('KONIEV, MRS.', 'RODGERS, MARIANNE', 0.0),
 ('KONIEV, MRS.', 'VALKIN', 0.0),
 ('KONIEV, MRS.', 'MUSE', 0.0),
 ('KONIEV, MRS.', 'BEETLE', 0.25),
 ('KONIEV, MRS.', 'OX', 0.0),
 ('KONIEV, MRS.', 'DISPROSIUM', 0.0),
 ('KONIEV, MRS.', 'PORTAL', 0.0),
 ('KONIEV, MRS.', 'MOONDRAGON DOPPELGAN', 0.16666666666666666),
 ('KONIEV, MRS.', 'MAN-BULL', 0.0),
 ('KONIEV, MRS.', 'RINGLET', 0.5),
 ('KONIEV, MRS.', 'STRIKER, SAM', 0.0),
 ('KONIEV, MRS.', 'DICHINSON, RICKY', 0.0),
 ('KONIEV, MRS.', 'VEDA', 0.0),
 ('KONIEV, MRS.', 'ASBESTOS LADY', 0.0),
 ('KONIEV, MRS.', 'WILD MAN', 0.0),
 ('KONIEV, MRS.', 'CARNAGE', 0.0),
 ('KONIEV, MRS.', 'DUNLEAVY, GEN. ARLEN', 0.0),
 ('KONIEV, MRS.', 'FRANKIE THE FISH', 0.0),
 ('KONIEV, MRS.', 'BITMAP', 0.0),
 ('KONIEV, MRS.', 'CABER', 0.0),
 ('KONIEV, MRS.', 'LAMENT', 0.5),
 ('KONIEV, MRS.', 'BLACK BRIGADE', 0.0),
 ('KONIEV, MRS.', 'HAWKSHAW', 0.0),
 ('KONIEV, MRS.', 'ANDROMEDA', 0.16666666666666666),
 ('KONIEV, MRS.', 'SISTER NIL', 0.0),
 ('KONIEV, MRS.', 'BALLANTINE, MRS.', 0.0),
 ('KONIEV, MRS.', 'MEGA MAX', 0.5),
 ('KONIEV, MRS.', 'JEFFRIES, DR.', 0.0),
 ('KONIEV, MRS.', 'SNOW, CHARLIE', 0.0),
 ('KONIEV, MRS.', 'CASSADA, ROBERTO', 0.0),
 ('KONIEV, MRS.', 'DEADBOLT', 1.0),
 ('KONIEV, MRS.', 'THUNDERSWORD', 0.0),
 ('KONIEV, MRS.', 'MAGIK', 0.3333333333333333),
 ('KONIEV, MRS.', 'LE FEY, MORGAN', 0.25),
 ('KONIEV, MRS.', 'SPIDER-MAN III', 0.0),
 ('KONIEV, MRS.', 'PSYKLOP', 0.0),
 ('KONIEV, MRS.', 'HARDY, STAN', 0.0),
 ('KONIEV, MRS.', 'WEIN, GLYNIS', 0.25),
 ('KONIEV, MRS.', 'HARD BOILED HENWY', 0.0),
 ('KONIEV, MRS.', 'TIMESLOT', 0.5),
 ('KONIEV, MRS.', 'LUMPKIN, WILLIAM WIL', 0.3333333333333333),
 ('KONIEV, MRS.', 'JINKU', 0.0),
 ('KONIEV, MRS.', 'ULTIMO', 0.0),
 ('KONIEV, MRS.', 'INFINITY', 0.0),
 ('KONIEV, MRS.', 'MAYE, APRIL', 0.0),
 ('KONIEV, MRS.', 'PHOENIX II', 0.5),
 ('KONIEV, MRS.', 'APE MAN II', 0.0),
 ('KONIEV, MRS.', 'MCGIVENEY, BULL', 0.0),
 ('KONIEV, MRS.', 'GHOST', 0.0),
 ('KONIEV, MRS.', 'NELSON, DEBBIE HARRI', 0.0),
 ('KONIEV, MRS.', 'VALKYRIE IV', 0.0),
 ('KONIEV, MRS.', 'RANDOM II', 0.5),
 ('KONIEV, MRS.', 'TOAD', 0.6666666666666666),
 ('KONIEV, MRS.', 'JACK FROST', 0.0),
 ('KONIEV, MRS.', 'BLACK RACER', 0.5),
 ('KONIEV, MRS.', 'TRAUMA', 0.0),
 ('KONIEV, MRS.', 'BAILEY, SARA GREY', 0.0),
 ('KONIEV, MRS.', 'SWEEPZWEAK', 0.5),
 ('KONIEV, MRS.', 'SYNCH', 0.5),
 ('KONIEV, MRS.', 'M', 1.0),
 ('KONIEV, MRS.', 'FRIGGA', 0.0),
 ('KONIEV, MRS.', 'NORTHSTAR DOPPELGANG', 0.0),
 ('KONIEV, MRS.', 'GAZELLE', 0.0),
 ('KONIEV, MRS.', 'NIGHTMARE', 0.16666666666666666),
 ('KONIEV, MRS.', 'WILLIAMS, MINDY', 0.0),
 ('KONIEV, MRS.', 'RAPTURE', 1.0),
 ('KONIEV, MRS.', 'DE LA SPIROSA, COUNT', 0.0),
 ('KONIEV, MRS.', 'FANDRAL [ASGARDIAN]', 0.0),
 ('KONIEV, MRS.', 'KYLUN', 0.0),
 ('KONIEV, MRS.', 'COLOSSUS', 0.0),
 ('KONIEV, MRS.', 'PORCUPINE', 0.0),
 ('KONIEV, MRS.', 'PROWLER', 0.0),
 ('KONIEV, MRS.', 'IKONN', 0.0),
 ('KONIEV, MRS.', 'TRAVELLER, DR. JUDAS', 0.0),
 ('KONIEV, MRS.', 'MORAN, KEVIN', 0.0),
 ('KONIEV, MRS.', 'UNION JACK II', 0.0),
 ('KONIEV, MRS.', 'RAGGADORR', 0.0),
 ('KONIEV, MRS.', 'HENRY, JOHN', 0.0),
 ('KONIEV, MRS.', 'TRAPSTER', 0.0),
 ('KONIEV, MRS.', 'JEMIAH', 0.0),
 ('KONIEV, MRS.', 'THORNN', 0.0),
 ('KONIEV, MRS.', 'DRAN, DAMON', 0.0),
 ('KONIEV, MRS.', 'CYCLONE', 0.0),
 ('KONIEV, MRS.', 'MILAN', 0.0),
 ('KONIEV, MRS.', 'MALICE IV', 1.0),
 ('KONIEV, MRS.', 'KRANG [ATLANTEAN]', 0.0),
 ('KONIEV, MRS.', 'DELGADO, FATHER', 0.0),
 ('KONIEV, MRS.', 'GEIGER', 0.0),
 ('KONIEV, MRS.', 'SANTINI, DR. JOSE', 0.0),
 ('KONIEV, MRS.', 'ERISHKIGEL II', 0.5),
 ('KONIEV, MRS.', 'LLYRON', 0.0),
 ('KONIEV, MRS.', 'OLSEN, JIMMY', 0.0),
 ('KONIEV, MRS.', "JACK O'LANTERN II", 0.0),
 ('KONIEV, MRS.', 'PUCK DOPPELGANGER II', 0.5),
 ('KONIEV, MRS.', 'SHADOW QUEEN', 0.0),
 ('KONIEV, MRS.', "T'KYLL ALABAR", 0.0),
 ('KONIEV, MRS.', 'FREE SPIRIT', 0.0),
 ('KONIEV, MRS.', 'BATTLE, MONTGOMERY', 0.0),
 ('KONIEV, MRS.', 'STACY, GWEN', 0.0),
 ('KONIEV, MRS.', 'FIELDING, CLARENCE', 0.0),
 ('KONIEV, MRS.', 'HATE', 0.25),
 ('KONIEV, MRS.', 'TWO-GUN KID', 0.0),
 ('KONIEV, MRS.', 'URTHONA', 0.0),
 ('KONIEV, MRS.', 'GLADIATOR II', 0.25),
 ('KONIEV, MRS.', 'PHOENIX 2013', 0.5),
 ('KONIEV, MRS.', 'MONGU', 0.0),
 ('KONIEV, MRS.', 'STONEWALL', 1.0),
 ('KONIEV, MRS.', 'SUNSHINE', 0.0),
 ('KONIEV, MRS.', 'ANTEUS', 0.5),
 ('KONIEV, MRS.', 'DEANGELO, DEVLIN', 0.0),
 ('KONIEV, MRS.', 'ZURAS', 0.0),
 ('KONIEV, MRS.', 'MAD VIKING', 0.0),
 ('KONIEV, MRS.', 'BROADSIDE', 0.5),
 ('KONIEV, MRS.', 'SENSO', 0.0),
 ('KONIEV, MRS.', 'INDECH', 0.0),
 ('KONIEV, MRS.', 'GULADKIN', 0.0),
 ('KONIEV, MRS.', 'KRAMER, STACEY', 0.0),
 ('KONIEV, MRS.', 'EHMAN, HEIDI', 0.0),
 ('KONIEV, MRS.', 'ZEMO', 0.0),
 ('KONIEV, MRS.', 'WALRUS', 0.0),
 ('KONIEV, MRS.', 'PRISM', 0.5),
 ('KONIEV, MRS.', 'RINTRAH', 0.0),
 ('KONIEV, MRS.', 'WHYTE, HARMONY (HARM', 0.5),
 ('KONIEV, MRS.', 'SUNFIRE', 0.14285714285714285),
 ('KONIEV, MRS.', 'MOONSTOMP', 0.0),
 ('KONIEV, MRS.', 'BROTHER ROYAL', 0.5),
 ('KONIEV, MRS.', 'FISK, VANESSA', 0.0),
 ('KONIEV, MRS.', 'ORINI', 0.0),
 ('KONIEV, MRS.', 'DRUIG', 0.0),
 ('KONIEV, MRS.', 'ATOR, GENERAL', 0.0),
 ('KONIEV, MRS.', 'TICKTOCK', 0.0),
 ('KONIEV, MRS.', 'ULTIMATOR', 0.0),
 ('KONIEV, MRS.', 'FOREARM', 0.5),
 ('KONIEV, MRS.', 'RIPTIDE', 1.0),
 ('KONIEV, MRS.', 'IKARIS', 0.16666666666666666),
 ('KONIEV, MRS.', 'PRETTY BOY', 1.0),
 ('KONIEV, MRS.', 'BAZIN, PHILLIPPE', 0.0),
 ('KONIEV, MRS.', 'CETHLANN', 0.0),
 ('KONIEV, MRS.', 'STANE, OBADIAH', 0.0),
 ('KONIEV, MRS.', 'MISS SACCHARINE', 0.5),
 ('KONIEV, MRS.', 'MOREAU, DAVID', 0.5),
 ('KONIEV, MRS.', 'ANDERSON, CHET', 0.0),
 ('KONIEV, MRS.', 'GORGON [INHUMAN]', 0.16666666666666666),
 ('KONIEV, MRS.', 'CYPHER', 0.4),
 ('KONIEV, MRS.', 'PARETSKY, DR. GARY', 0.0),
 ('KONIEV, MRS.', 'KLEINSTOCK, SVEN', 0.3333333333333333),
 ('KONIEV, MRS.', 'ALFY', 0.5),
 ('KONIEV, MRS.', 'SENYAKA', 0.0),
 ('KONIEV, MRS.', 'REQUIEM', 0.5),
 ('KONIEV, MRS.', 'KOLE, JOHN', 0.5),
 ('KONIEV, MRS.', 'SCARECROW II', 0.0),
 ('KONIEV, MRS.', 'NOVA', 0.2857142857142857),
 ('KONIEV, MRS.', 'STANCHEK, DR.', 0.0),
 ('KONIEV, MRS.', 'KESTRUS', 0.0),
 ('KONIEV, MRS.', 'SZARDOS, STEFAN', 0.5),
 ('KONIEV, MRS.', 'MICAWBER, TRISTRAM', 0.0),
 ('KONIEV, MRS.', 'MOREAU, PHILLIP', 0.5),
 ('KONIEV, MRS.', 'CONAN, NEAL', 0.5),
 ('KONIEV, MRS.', 'TONGAH', 0.0),
 ('KONIEV, MRS.', 'ROSEN, WALTER', 0.0),
 ('KONIEV, MRS.', 'MORRELL, SABRINA', 0.5),
 ('KONIEV, MRS.', 'GUNTHAR II', 0.0),
 ('KONIEV, MRS.', 'PRINCESS PYTHON', 0.0),
 ('KONIEV, MRS.', 'EL GIBAR, ACHMED', 0.0),
 ('KONIEV, MRS.', 'THOMPSON, JESSICA JE', 0.0),
 ('KONIEV, MRS.', 'SCAR THE STALKER', 0.0),
 ('KONIEV, MRS.', 'KYMRI', 0.5),
 ('KONIEV, MRS.', 'HEIMDALL [ASGARDIAN]', 0.0),
 ('KONIEV, MRS.', 'SLAMBEAUX', 0.0),
 ('KONIEV, MRS.', 'EARLE, CARYN', 0.0),
 ('KONIEV, MRS.', 'CHONDU', 0.0),
 ('KONIEV, MRS.', 'MICHEL, RAFE', 0.0),
 ('KONIEV, MRS.', 'REM-RAM', 0.5),
 ('KONIEV, MRS.', 'AGAMOTTO', 0.0),
 ('KONIEV, MRS.', 'SUGAR MAN', 1.0),
 ('KONIEV, MRS.', 'CALDRONE, JAVIER', 0.0),
 ('KONIEV, MRS.', 'THUNDERBIRD', 1.0),
 ('KONIEV, MRS.', 'MAN-APE', 0.0),
 ('KONIEV, MRS.', 'TOTEM', 0.0),
 ('KONIEV, MRS.', 'SWARM', 0.0),
 ('KONIEV, MRS.', 'NALES, DET. REBECCA ', 0.0),
 ('KONIEV, MRS.', 'JAYE-FIRESTONE, GING', 0.0),
 ('KONIEV, MRS.', 'SKIN II', 0.0),
 ('KONIEV, MRS.', 'BLAZE, ROXANNE SIMPS', 0.0),
 ('KONIEV, MRS.', 'SNOW, TANDY', 0.0),
 ('KONIEV, MRS.', 'COBALT MAN', 0.0),
 ('KONIEV, MRS.', 'BATWING', 0.0),
 ('KONIEV, MRS.', 'ARMBRUSTER, COLONEL ', 0.3333333333333333),
 ('KONIEV, MRS.', 'GORGILLA', 0.0),
 ('KONIEV, MRS.', 'SIN-EATER', 0.0),
 ('KONIEV, MRS.', 'SAMEDAR', 1.0),
 ('KONIEV, MRS.', 'APOLLO [GREEK GOD]', 0.0),
 ('KONIEV, MRS.', 'JUTTA', 0.5),
 ('KONIEV, MRS.', 'RECORDER II', 0.0),
 ('KONIEV, MRS.', 'PARKER, BEN', 0.0),
 ('KONIEV, MRS.', 'LETHA', 0.4),
 ('KONIEV, MRS.', 'APOCALYPSE', 0.4),
 ('KONIEV, MRS.', 'MASTER KHAN', 0.0),
 ('KONIEV, MRS.', 'BLIZZARD', 0.0),
 ('KONIEV, MRS.', 'OKOYE', 0.0),
 ('KONIEV, MRS.', 'PRESTER JOHN', 0.0),
 ('KONIEV, MRS.', 'SUNDER', 0.2),
 ('KONIEV, MRS.', 'BLOODSHED', 0.0),
 ('KONIEV, MRS.', 'GAUNTLET', 1.0),
 ('KONIEV, MRS.', 'BLACK, CYRUS', 0.0),
 ('KONIEV, MRS.', 'BLUE STREAK', 0.0),
 ('KONIEV, MRS.', 'MR. TERMINEUS', 0.0),
 ('KONIEV, MRS.', 'MAJOR DOMO', 0.6666666666666666),
 ('KONIEV, MRS.', 'UROC', 0.0),
 ('KONIEV, MRS.', 'DEFENSOR', 0.3333333333333333),
 ('KONIEV, MRS.', 'JONES, LOUISE', 0.3333333333333333),
 ('KONIEV, MRS.', 'ANCESTOR', 0.0),
 ('KONIEV, MRS.', 'LUCIFER', 0.0),
 ('KONIEV, MRS.', 'RED GUARDIAN IV', 0.0),
 ('KONIEV, MRS.', 'CENTURY', 0.0),
 ('KONIEV, MRS.', 'CRIMSON CRUSADER', 0.5),
 ('KONIEV, MRS.', 'VOSTOK', 0.25),
 ('KONIEV, MRS.', 'FANCY DAN', 0.0),
 ('KONIEV, MRS.', 'MARTINEZ, ALITHA', 0.0),
 ('KONIEV, MRS.', 'RORY, RICHARD', 0.0),
 ('KONIEV, MRS.', 'SHATTERFIST', 0.0),
 ('KONIEV, MRS.', 'BRAIN CELL', 0.5),
 ('KONIEV, MRS.', 'WIZARD', 0.14285714285714285),
 ('KONIEV, MRS.', "O'DONNELL, EAMON", 1.0),
 ('KONIEV, MRS.', 'SPIDERCIDE', 0.0),
 ('KONIEV, MRS.', 'SENTRY II', 0.0),
 ('KONIEV, MRS.', 'VOLLA', 0.0),
 ('KONIEV, MRS.', 'EPOCH', 0.2857142857142857),
 ('KONIEV, MRS.', 'MALUS, DR. KARL', 0.0),
 ('KONIEV, MRS.', 'BALABAN', 0.0),
 ('KONIEV, MRS.', 'MANN, DR. J. VERNON ', 0.0),
 ('KONIEV, MRS.', 'RUSSELL, LISSA', 0.0),
 ('KONIEV, MRS.', 'SCORPION', 0.0),
 ('KONIEV, MRS.', "Q'WAKE", 0.0),
 ('KONIEV, MRS.', 'WAR', 1.0),
 ('KONIEV, MRS.', 'DORMAMMU', 0.0),
 ('KONIEV, MRS.', 'MR. JUSTICE', 0.0),
 ('KONIEV, MRS.', 'TORO', 0.0),
 ('KONIEV, MRS.', 'CUSHING, KATE', 0.3333333333333333),
 ('KONIEV, MRS.', 'GLADIATOR', 0.0),
 ('KONIEV, MRS.', 'SANGUINE', 1.0),
 ('KONIEV, MRS.', 'CONTONI, PAUL', 0.0),
 ('KONIEV, MRS.', 'SCORCH II', 0.5),
 ('KONIEV, MRS.', 'GLABOO', 0.0),
 ('KONIEV, MRS.', 'HYPERTRON II', 0.0),
 ('KONIEV, MRS.', 'RASPUTIN, MIKHAIL', 0.0),
 ('KONIEV, MRS.', 'PROMETHEUS [OLYMPIAN', 0.0),
 ('KONIEV, MRS.', 'JESTER III', 0.0),
 ('KONIEV, MRS.', 'MINDWORM', 0.0),
 ('KONIEV, MRS.', 'KRAKEN', 0.0),
 ('KONIEV, MRS.', 'RATTLER', 0.25),
 ('KONIEV, MRS.', 'YUKIO', 1.0),
 ('KONIEV, MRS.', 'VULTURE', 0.125),
 ('KONIEV, MRS.', 'PESTILENCE', 0.0),
 ('KONIEV, MRS.', 'GEIRRODUR', 0.0),
 ('KONIEV, MRS.', 'THUNDERSTRIKE', 0.16666666666666666),
 ('KONIEV, MRS.', 'PSYLOCKE', 0.2857142857142857),
 ('KONIEV, MRS.', 'FOOL', 0.0),
 ('KONIEV, MRS.', 'BUSHKIN, BARNEY', 0.0),
 ('KONIEV, MRS.', 'THOMAS, COMMANDER DA', 0.0),
 ('KONIEV, MRS.', 'UNDERTAKER', 0.0),
 ('KONIEV, MRS.', 'RAMBEAU, FRANK', 0.0),
 ('KONIEV, MRS.', 'CAPTAIN BARRACUDA', 0.0),
 ('KONIEV, MRS.', 'BONEBREAKER', 0.0),
 ('KONIEV, MRS.', 'CRUX', 0.5),
 ('KONIEV, MRS.', 'HIGGINS, LINDA', 0.5),
 ('KONIEV, MRS.', 'KAISHEK, GOM', 0.5),
 ('KONIEV, MRS.', 'RALEIGH, RICHARD', 0.0),
 ('KONIEV, MRS.', 'REAPER II', 0.5),
 ('KONIEV, MRS.', 'CAPTAIN AMERICA IV', 0.0),
 ('KONIEV, MRS.', 'DR. OCTOPUS', 0.4),
 ('KONIEV, MRS.', 'VIBRO', 0.0),
 ('KONIEV, MRS.', 'BARON STRUCKER', 0.0),
 ('KONIEV, MRS.', 'GUARDSMAN', 0.0),
 ('KONIEV, MRS.', 'SPACE PHANTOM', 0.0),
 ('KONIEV, MRS.', '4-D MAN', 0.0),
 ('KONIEV, MRS.', 'WARFIST', 0.0),
 ('KONIEV, MRS.', 'LADY LARK', 0.0),
 ('KONIEV, MRS.', 'ALEXANDER, CARRIE', 0.0),
 ('KONIEV, MRS.', 'IRIDIA [INHUMAN]', 0.0),
 ('KONIEV, MRS.', 'WATSON-PARKER, MARY ', 0.3333333333333333),
 ('KONIEV, MRS.', 'WATSON, PHILIP', 0.0),
 ('KONIEV, MRS.', 'SILVER SURFER', 0.14285714285714285),
 ('KONIEV, MRS.', 'SPRAGG', 0.0),
 ('KONIEV, MRS.', 'SPIRAL', 0.5),
 ('KONIEV, MRS.', 'RAX', 1.0),
 ('KONIEV, MRS.', 'DRAGONFLY II', 0.0),
 ('KONIEV, MRS.', "K'RK [SHI'AR]", 0.5),
 ('KONIEV, MRS.', 'DEZOAN, JOHN', 0.0),
 ('KONIEV, MRS.', 'HOGGOTH', 0.0),
 ('KONIEV, MRS.', 'VENUS', 0.0),
 ('KONIEV, MRS.', 'LARROCA, SALVADORE', 0.0),
 ('KONIEV, MRS.', 'BALDWIN, JUSTIN', 0.0),
 ('KONIEV, MRS.', 'HECTOR II', 0.0),
 ('KONIEV, MRS.', 'IRON MAN', 0.0021953896816684962),
 ('KONIEV, MRS.', 'MAGNUS', 0.0),
 ('KONIEV, MRS.', 'BENNETT, MITCH', 0.0),
 ('KONIEV, MRS.', 'HELLSTROM, VICTORIA', 0.0),
 ('KONIEV, MRS.', 'PARKER, MAY | TIMESL', 0.0),
 ('KONIEV, MRS.', 'THUG', 0.0),
 ('KONIEV, MRS.', 'BLACK CAT', 0.0),
 ('KONIEV, MRS.', 'GHOST RIDER III', 0.16666666666666666),
 ('KONIEV, MRS.', 'CORD, EDWIN', 0.0),
 ('KONIEV, MRS.', 'ZODIAK', 0.0),
 ('KONIEV, MRS.', 'JONES, NELL', 0.0),
 ('KONIEV, MRS.', 'KOSINSKI, HARLEY', 0.0),
 ('KONIEV, MRS.', 'TURBO II (A)', 0.0),
 ('KONIEV, MRS.', 'KARKAS [DEVIANT]', 0.0),
 ('KONIEV, MRS.', 'MULDOON, SGT. BRIAN', 0.0),
 ('KONIEV, MRS.', 'WATSON, KRISTY', 0.0),
 ('KONIEV, MRS.', 'HALE, TOM', 0.0),
 ('KONIEV, MRS.', 'MANTA', 0.5),
 ('KONIEV, MRS.', "CH'THON", 0.0),
 ('KONIEV, MRS.', 'TAYLOR, JACK', 0.0),
 ('KONIEV, MRS.', 'HITLER, ADOLF', 0.0),
 ('KONIEV, MRS.', 'TAKU', 0.0),
 ('KONIEV, MRS.', 'STYGLUT', 0.5),
 ('KONIEV, MRS.', 'BANSHEE', 0.4),
 ('KONIEV, MRS.', 'CRIMSON COWL II', 0.0),
 ('KONIEV, MRS.', 'JOHNSMEYER, JAMES', 0.0),
 ('KONIEV, MRS.', 'NICHOLS, LORNA', 0.0),
 ('KONIEV, MRS.', 'GYPSY MOTH', 0.125),
 ('KONIEV, MRS.', 'KELLY, SEN. ROBERT E', 0.25),
 ('KONIEV, MRS.', 'SEA URCHIN', 0.0),
 ('KONIEV, MRS.', 'SCORPIO III', 0.5),
 ('KONIEV, MRS.', 'SPHINX', 0.0),
 ('KONIEV, MRS.', 'CORRIGAN, ARCHIBALD ', 0.5),
 ('KONIEV, MRS.', 'WONDER MAN', 0.25),
 ('KONIEV, MRS.', 'HOGAN, HAROLD J. HAP', 0.0),
 ('KONIEV, MRS.', 'EVANS, DORIS', 0.0),
 ('KONIEV, MRS.', 'THINKER', 0.2857142857142857),
 ('KONIEV, MRS.', 'HARVEST', 0.0),
 ('KONIEV, MRS.', 'FIZ [SKRULL]', 1.0),
 ('KONIEV, MRS.', "O'DONNELL", 0.3333333333333333),
 ('KONIEV, MRS.', 'RED WOLF III', 0.2857142857142857),
 ('KONIEV, MRS.', 'ROSSI, COL. MICHAEL ', 1.0),
 ('KONIEV, MRS.', 'HOGUN [ASGARDIAN]', 0.0),
 ('KONIEV, MRS.', 'CAMPBELL, MEREDITH', 0.0),
 ('KONIEV, MRS.', 'DOLLAR BILL', 0.0),
 ('KONIEV, MRS.', 'HOARD, JOSH', 0.0),
 ('KONIEV, MRS.', 'BLIND AL', 0.0),
 ('KONIEV, MRS.', 'PENNYWORTH, J.C.', 0.0),
 ('KONIEV, MRS.', 'CRIMSON COMMANDO', 0.3333333333333333),
 ('KONIEV, MRS.', 'WHIPLASH', 0.3333333333333333),
 ('KONIEV, MRS.', 'ESON [CELESTIAL]', 0.0),
 ('KONIEV, MRS.', 'VOLSTAGG', 0.0),
 ('KONIEV, MRS.', 'MOOMJI, INDRIES', 0.0),
 ('KONIEV, MRS.', 'DEATH', 0.3333333333333333),
 ('KONIEV, MRS.', 'ICON', 0.0),
 ('KONIEV, MRS.', 'HRIMHARI', 0.5),
 ('KONIEV, MRS.', 'STILETTO', 0.0),
 ('KONIEV, MRS.', 'SNOWBIRD', 0.25),
 ('KONIEV, MRS.', 'MAGRUDER, ALLIE', 0.2),
 ('KONIEV, MRS.', 'CHANEY, AMANDA', 0.0),
 ('KONIEV, MRS.', 'PALADIN', 0.0),
 ('KONIEV, MRS.', 'EJULP', 1.0),
 ('KONIEV, MRS.', 'BARBARUS', 0.4),
 ('KONIEV, MRS.', 'THOMAS, ROY', 0.0),
 ('KONIEV, MRS.', 'EXECUTIONER II', 0.0),
 ('KONIEV, MRS.', 'JONES, HUGH', 0.0),
 ('KONIEV, MRS.', 'MS. FORTUNE', 0.0),
 ('KONIEV, MRS.', 'SLEEPER', 0.0),
 ('KONIEV, MRS.', 'OSIRIS', 0.0),
 ('KONIEV, MRS.', 'JUVAN', 0.0),
 ('KONIEV, MRS.', 'SAGITTARIUS', 0.0),
 ('KONIEV, MRS.', 'SUN, LIN', 0.0),
 ('KONIEV, MRS.', 'MR. NACHT', 0.0),
 ('KONIEV, MRS.', 'APPALLA', 0.0),
 ('KONIEV, MRS.', 'ROSS, COURTNEY', 0.0),
 ('KONIEV, MRS.', 'PURSUER', 0.0),
 ('KONIEV, MRS.', 'GOTH', 1.0),
 ('KONIEV, MRS.', 'AVIOS', 0.5),
 ('KONIEV, MRS.', 'NOMAD III', 0.0),
 ('KONIEV, MRS.', 'METEORITE III', 0.0),
 ('KONIEV, MRS.', 'LONGSHOT', 0.6666666666666666),
 ('KONIEV, MRS.', 'BROWN, FRANK', 0.0),
 ('KONIEV, MRS.', 'SENSATIONAL HYDRA', 0.0),
 ('KONIEV, MRS.', 'JACKSON, ANDREW JOCK', 0.0),
 ('KONIEV, MRS.', 'CEREBRITE ALPHA', 0.5),
 ('KONIEV, MRS.', 'PAM', 0.5),
 ('KONIEV, MRS.', 'MODRED THE MYSTIC', 0.0),
 ('KONIEV, MRS.', 'BRIDGE, GEORGE WASHI', 0.16666666666666666),
 ('KONIEV, MRS.', 'PRINCESS FEN [ATLANT', 0.0),
 ('KONIEV, MRS.', 'ZELDA', 0.5),
 ('KONIEV, MRS.', 'FIREFIST', 0.2),
 ('KONIEV, MRS.', 'BERGLUND, GEN.', 0.0),
 ('KONIEV, MRS.', 'SMASHER IV', 0.3333333333333333),
 ('KONIEV, MRS.', 'LIZARD', 0.5),
 ('KONIEV, MRS.', 'DAREDEVIL DOPPELGANG', 0.0),
 ('KONIEV, MRS.', 'ANAMI, HIROSHI', 0.0),
 ('KONIEV, MRS.', 'CLEA', 0.4),
 ('KONIEV, MRS.', 'DOGGEREL', 0.0),
 ('KONIEV, MRS.', 'LOCKLIN, GENERAL MER', 0.0),
 ('KONIEV, MRS.', 'DARKHAWK', 0.4),
 ('KONIEV, MRS.', 'QNAX', 0.0),
 ('KONIEV, MRS.', 'MISFIT', 0.0),
 ('KONIEV, MRS.', 'PARTRIDGE, STEVE', 0.0),
 ('KONIEV, MRS.', 'MERLYN', 0.0),
 ('KONIEV, MRS.', 'NIGHTHAWK II', 0.0),
 ('KONIEV, MRS.', 'SPRITE', 0.0),
 ('KONIEV, MRS.', 'ECCE [WATCHER]', 0.0),
 ('KONIEV, MRS.', 'HYPERION', 0.0),
 ('KONIEV, MRS.', 'FAWKES', 0.0),
 ('KONIEV, MRS.', 'PAGE, KAREN', 0.125),
 ('KONIEV, MRS.', 'THERMO', 0.0),
 ('KONIEV, MRS.', 'PIP', 0.3333333333333333),
 ('KONIEV, MRS.', 'AURORA DOPPELGANGER', 0.0),
 ('KONIEV, MRS.', 'PRIMUS', 0.0),
 ('KONIEV, MRS.', 'PUNCHOUT', 0.5),
 ('KONIEV, MRS.', 'SLICK', 0.0),
 ('KONIEV, MRS.', 'DENAK', 0.0),
 ('KONIEV, MRS.', 'GREEN, LILA', 0.0),
 ('KONIEV, MRS.', 'MASTER LINK', 0.0),
 ('KONIEV, MRS.', 'REAGAN, RONALD', 0.5),
 ('KONIEV, MRS.', 'FIFOLET', 0.5),
 ('KONIEV, MRS.', 'LIVING MONOLITH', 0.5),
 ('KONIEV, MRS.', 'ARMAGEDDON', 0.0),
 ('KONIEV, MRS.', 'MARRS, DESMOND', 0.0),
 ('KONIEV, MRS.', 'MEDUSA', 0.16666666666666666),
 ('KONIEV, MRS.', 'CALLAGHAN, JAMES', 0.0),
 ('KONIEV, MRS.', 'JARELLA', 0.0),
 ('KONIEV, MRS.', 'FILLMORE, SHERIFF DO', 0.0),
 ('KONIEV, MRS.', 'SINGER', 0.5),
 ('KONIEV, MRS.', 'SPOOR', 0.0),
 ('KONIEV, MRS.', 'SEDNA', 0.0),
 ('KONIEV, MRS.', 'RANGER I', 0.0),
 ('KONIEV, MRS.', 'ELATHAN', 0.0),
 ('KONIEV, MRS.', 'LEEDS, NED', 0.4),
 ('KONIEV, MRS.', "CH'OD", 0.4),
 ('KONIEV, MRS.', 'ZIRAN', 0.2),
 ('KONIEV, MRS.', 'MACENDALE, JASON JR.', 0.0),
 ('KONIEV, MRS.', 'MASTERS, ALICIA REIS', 0.14285714285714285),
 ('KONIEV, MRS.', 'REEVES, MAJOR', 0.0),
 ('KONIEV, MRS.', 'OUTRIDER II', 0.0),
 ('KONIEV, MRS.', 'SABRA', 0.14285714285714285),
 ('KONIEV, MRS.', 'MCDOWELL, SCOTTY', 0.0),
 ('KONIEV, MRS.', 'ST. LAWRENCE, COL. C', 0.0),
 ('KONIEV, MRS.', 'NIGHT RIDER', 0.0),
 ('KONIEV, MRS.', 'CHADWICK, DR. HESPER', 0.0),
 ('KONIEV, MRS.', 'LEWIS, SHIRLEY WASHI', 0.0),
 ('KONIEV, MRS.', 'MAYHEM', 0.0),
 ('KONIEV, MRS.', 'MEER, MAX', 0.0),
 ('KONIEV, MRS.', 'COLE, WADE', 1.0),
 ('KONIEV, MRS.', 'COOPER, DR. VALERIE', 0.3333333333333333),
 ('KONIEV, MRS.', 'MR. IMMORTAL', 0.0),
 ('KONIEV, MRS.', 'SIBERCAT', 0.0),
 ('KONIEV, MRS.', 'LEE, STAN', 0.2),
 ('KONIEV, MRS.', 'BLACK TOM CASSIDY', 1.0),
 ('KONIEV, MRS.', 'KHORYPHOS [ETERNAL]', 0.0),
 ('KONIEV, MRS.', 'FIREBUG', 0.3333333333333333),
 ('KONIEV, MRS.', 'NORRISS, JACK', 0.0),
 ('KONIEV, MRS.', 'CRIPPLER', 0.0),
 ('KONIEV, MRS.', 'JACK OF HEARTS', 0.14285714285714285),
 ('KONIEV, MRS.', 'SLYDE', 0.0),
 ('KONIEV, MRS.', 'SAWYER, GENERAL SAMU', 0.0),
 ('KONIEV, MRS.', 'X-MAN', 0.3333333333333333),
 ('KONIEV, MRS.', 'SVAROG', 0.2),
 ('KONIEV, MRS.', 'REINHOLDT', 0.0),
 ('KONIEV, MRS.', 'LIBRA', 0.0),
 ('KONIEV, MRS.', 'SIRYN', 0.5),
 ('KONIEV, MRS.', 'WARGOD', 0.0),
 ('KONIEV, MRS.', 'BOOMSLANG', 0.3333333333333333),
 ('KONIEV, MRS.', 'DEATHUNT 9000', 0.0),
 ('KONIEV, MRS.', 'OLSON, MRS.', 0.0),
 ('KONIEV, MRS.', 'KRONOS', 0.16666666666666666),
 ('KONIEV, MRS.', 'JONES, JANIS', 0.3333333333333333),
 ('KONIEV, MRS.', 'QUASIMODO', 0.0),
 ('KONIEV, MRS.', 'WARHAWK II', 1.0),
 ('KONIEV, MRS.', 'ROUST', 0.0),
 ('KONIEV, MRS.', 'MEDEA', 0.0),
 ('KONIEV, MRS.', 'ATTUMA', 0.25),
 ('KONIEV, MRS.', 'VENOM', 0.3333333333333333),
 ('KONIEV, MRS.', 'CRUCIBLE', 0.0),
 ('KONIEV, MRS.', 'GAROKK', 1.0),
 ('KONIEV, MRS.', 'NOCTURNE II', 0.0),
 ('KONIEV, MRS.', 'DOMINA', 0.5),
 ('KONIEV, MRS.', 'RED RAVEN', 0.0),
 ('KONIEV, MRS.', 'MORGAN, JERROLD JERR', 0.0),
 ('KONIEV, MRS.', 'KANE, SUGAR', 0.5),
 ('KONIEV, MRS.', "O'BRYAN, BOB", 0.0),
 ('KONIEV, MRS.', 'MISSING LINK II', 0.0),
 ('KONIEV, MRS.', 'SISTER DEATH', 0.0),
 ('KONIEV, MRS.', 'EROS', 0.0),
 ('KONIEV, MRS.', 'THENA', 0.0),
 ('KONIEV, MRS.', 'SPLICE', 0.0),
 ('KONIEV, MRS.', 'LASHER', 0.0),
 ('KONIEV, MRS.', 'MINERVA [KREE]', 0.0),
 ('KONIEV, MRS.', 'CALUMET', 0.0),
 ('KONIEV, MRS.', 'WOLFSBANE', 0.16666666666666666),
 ('KONIEV, MRS.', 'HORTON, PROFESSOR PH', 0.0),
 ('KONIEV, MRS.', 'BEAUTY', 0.0),
 ('KONIEV, MRS.', 'GAUNT, OFFICER KIM', 0.0),
 ('KONIEV, MRS.', 'LORD LEADER', 0.0),
 ('KONIEV, MRS.', 'SPEEDBALL DOPPELGANG', 0.5),
 ('KONIEV, MRS.', 'NIGHT PHANTOM', 0.0),
 ('KONIEV, MRS.', 'GREEN GOBLIN', 0.125),
 ('KONIEV, MRS.', 'BOUDREAUX, BELLA DON', 1.0),
 ('KONIEV, MRS.', 'CREED, GRAYDON', 0.6666666666666666),
 ('KONIEV, MRS.', 'MAN-MOUNTAIN MARKO', 0.0),
 ('KONIEV, MRS.', 'JANSON, DR. ANDREA', 0.0),
 ('KONIEV, MRS.', 'DRAGONFLY', 0.6666666666666666),
 ('KONIEV, MRS.', 'ROSE III', 0.0),
 ('KONIEV, MRS.', 'DEMOGOBLIN', 0.0),
 ('KONIEV, MRS.', 'VOLCANA', 0.5),
 ('KONIEV, MRS.', 'MOONDARK', 0.0),
 ('KONIEV, MRS.', 'LIVING LASER', 0.0),
 ('KONIEV, MRS.', 'LOCUST', 0.0),
 ('KONIEV, MRS.', 'STYX', 0.0),
 ('KONIEV, MRS.', 'MARVEL GIRL', 0.125),
 ('KONIEV, MRS.', 'NORRISS, SISTER BARB', 0.25),
 ('KONIEV, MRS.', 'MONOCLE', 0.0),
 ('KONIEV, MRS.', 'HOGAN, VIRGINIA PEPP', 0.0),
 ('KONIEV, MRS.', 'TOLLIVER, EMMA', 0.0),
 ('KONIEV, MRS.', 'BATTLEAXE II', 0.5),
 ('KONIEV, MRS.', 'AGEE, DR. AUBREY', 0.5),
 ('KONIEV, MRS.', 'POWER, MARGARET', 0.25),
 ('KONIEV, MRS.', 'PIERCE, ALEXANDER GO', 0.16666666666666666),
 ('KONIEV, MRS.', 'ARTEMIS', 0.0),
 ('KONIEV, MRS.', 'VISION DOPPELGANGER', 0.4),
 ('KONIEV, MRS.', 'HESCAMAR', 0.0),
 ('KONIEV, MRS.', 'WOO, JIMMY', 0.0),
 ('KONIEV, MRS.', 'MADAME MASQUE VI', 0.0),
 ('KONIEV, MRS.', 'DRUMM, COLLETTE', 0.0),
 ('KONIEV, MRS.', 'RINGMASTER', 0.0),
 ('KONIEV, MRS.', 'BROKER', 0.0),
 ('KONIEV, MRS.', 'PRIME MOVER', 0.0),
 ('KONIEV, MRS.', 'NIGHTSIDE', 0.0),
 ('KONIEV, MRS.', 'BLAIRE, CARTER', 0.25),
 ('KONIEV, MRS.', 'REDEEMER', 0.0),
 ('KONIEV, MRS.', 'WIDOW-MAKER', 0.0),
 ('KONIEV, MRS.', 'DEL HIERRO, BLANCA', 0.0),
 ('KONIEV, MRS.', 'TITANIA II', 0.125),
 ('KONIEV, MRS.', 'CANCELLATOR', 0.0),
 ('KONIEV, MRS.', 'AHURA', 0.0),
 ('KONIEV, MRS.', 'PROFESSOR GIBBON', 0.5),
 ('KONIEV, MRS.', 'SUMMERS, MADELYNE MA', 0.25),
 ('KONIEV, MRS.', 'TRITON DOPPELGANGER', 0.0),
 ('KONIEV, MRS.', 'FORTUNATO', 0.0),
 ('KONIEV, MRS.', 'GATECRASHER', 0.0),
 ('KONIEV, MRS.', 'NECROMANCER', 0.0),
 ('KONIEV, MRS.', 'MENTOR', 0.3333333333333333),
 ('KONIEV, MRS.', 'BLACKWING', 0.0),
 ('KONIEV, MRS.', 'PHASTOS [ETERNAL]', 0.0),
 ('KONIEV, MRS.', 'CODY, GARY', 1.0),
 ('KONIEV, MRS.', 'JONES, MARLO CHANDLE', 0.0),
 ('KONIEV, MRS.', 'CROWE, GERONIMO', 0.5),
 ('KONIEV, MRS.', 'KANE, MARCY', 0.0),
 ('KONIEV, MRS.', 'POLESTAR', 0.0),
 ('KONIEV, MRS.', 'BYRRAH [ATLANTEAN]', 0.0),
 ('KONIEV, MRS.', 'ROSS, GEN. THADDEUS ', 0.4),
 ('KONIEV, MRS.', 'RUIZ, RIGGER', 0.0),
 ('KONIEV, MRS.', 'SONDHEIM, ERICA', 0.0),
 ('KONIEV, MRS.', 'STRUCKER, ANDREAS', 0.25),
 ('KONIEV, MRS.', 'SANDHURST, VINCENT', 0.0),
 ('KONIEV, MRS.', 'MINDSINGER', 0.0),
 ('KONIEV, MRS.', 'TINKER, SHEA', 0.0),
 ('KONIEV, MRS.', 'RIOT', 0.0),
 ('KONIEV, MRS.', 'SILVERCLAW', 0.25),
 ('KONIEV, MRS.', 'BENNETT, DANNY', 0.0),
 ('KONIEV, MRS.', 'JOHNSON', 0.0),
 ('KONIEV, MRS.', 'HOUSTON, STELLA', 0.0),
 ('KONIEV, MRS.', 'NOMAD II', 0.0),
 ('KONIEV, MRS.', 'PROTEUS II', 1.0),
 ('KONIEV, MRS.', 'DIABLO', 0.0),
 ('KONIEV, MRS.', 'TIWAZ', 0.0),
 ('KONIEV, MRS.', 'SLITHER', 0.0),
 ('KONIEV, MRS.', 'PILEDRIVER II', 0.4),
 ('KONIEV, MRS.', 'DEMONICUS', 0.0),
 ('KONIEV, MRS.', 'BLOODSTONE', 0.0),
 ('KONIEV, MRS.', 'BADALAMENTE, JESUS', 0.5),
 ('KONIEV, MRS.', 'TOMAZOOMA', 0.0),
 ('KONIEV, MRS.', 'INFERNO II', 0.0),
 ('KONIEV, MRS.', 'BAXTER, NOAH', 0.0),
 ('KONIEV, MRS.', 'CANDRA', 0.5),
 ('KONIEV, MRS.', 'CAPTAIN AMERICA DOPP', 0.16666666666666666),
 ('KONIEV, MRS.', 'HINKLEY, ADMIRAL', 0.0),
 ('KONIEV, MRS.', 'FACADE', 0.0),
 ('KONIEV, MRS.', 'SCHWINNER, DR. ERIC', 0.0),
 ('KONIEV, MRS.', 'MADAME MASQUE', 0.0),
 ('KONIEV, MRS.', 'BOMBSHELL', 0.0),
 ('KONIEV, MRS.', 'BLAQUESMITH', 0.3333333333333333),
 ('KONIEV, MRS.', 'YEAGER, CALLIE', 0.0),
 ('KONIEV, MRS.', 'TODD, FISHER', 0.0),
 ...]

Adamic-Adar ¶

In [58]:
%%time
# Prediction using Adamic Adar 
pred_adamic = list(nx.adamic_adar_index(Gb_train))
score_adamic, label_adamic = zip(*[(s, (u,v) in edge_subset) for (u,v,s) in pred_adamic])

# Compute the ROC AUC Score
fpr_adamic, tpr_adamic, _ = roc_curve(label_adamic, score_adamic)
auc_adamic = round(roc_auc_score(label_adamic, score_adamic), 4)

CPU times: total: 3min
Wall time: 3min

Preferential Attachment ¶

In [59]:
%%time
# Compute the Preferential Attachment
pred_pref = list(nx.preferential_attachment(Gb_train))
score_pref, label_pref = zip(*[(s, (u,v) in edge_subset) for (u,v,s) in pred_pref])

fpr_pref, tpr_pref, _ = roc_curve(label_pref, score_pref)
auc_pref = round(roc_auc_score(label_pref, score_pref), 4)

CPU times: total: 2min 36s
Wall time: 2min 36s

Plotting the results ¶

In [60]:
plt.figure(figsize=(18,12))
plt.plot(fpr_jaccard, tpr_jaccard, label="jaccard :"+str(auc_jaccard))
plt.plot(fpr_pref, tpr_pref, label="adamic :"+str(auc_adamic))
plt.plot(fpr_adamic, tpr_adamic, label="pref :     "+str(auc_pref))
plt.plot([0, 1], [0, 1], "k--", alpha = 0.3)
plt.ylabel('True Positive Rate')
plt.xlabel('False Positive Rate')
plt.legend()
plt.show()

Top

Afterthought ¶

Which Algo? ¶

Both methods show the possibility of predicting links in a network graph with predicting power in the range of 63-78%. The results aren't optimal but are respectable. The classification method dropped 7% in the accuracy against the test data to 63%, showing there is overfitting. Believe it is likely due to multicollinearity between the features. The AUC for Jaccard was subpar while Adamic seemed promising at 0.7636 but the plot was odd. Preferential Attachment performed well at 0.7793.

Keeping in mind that both used sample dataset of 8 popular heroes, where links of these 8 heroes were extracted. The 2nd degree links where possible links between other heroes were left out. This is particularly obvious in the network graph depicting many isolated nodes due to random removal of links. If there 2nd and 3rd degree links were included, the network is likely to provide more comprehensive information.

Using link prediction in comic heroes may not be ideal compared to social network analysis. In social network, there are latent relationship between the nodes. Eg links maybe due to hobby, religion, etc. As for comics heroes, it is difficult to imagine latent relationships between the characters other than the creativity of the script writers.

What's next? ¶

1) Use the full dataset rather than the extracted sample for further experimentation. The outcome may yield better results where 2nd and 3rd degree links are present. However computational requirement will explode.

2) Further finetune the classification parameters.

3) Look into Graph Neural Network.

@End